Region segmentation method and device for hearing aid shell three-dimensional model and medium

Abstract

The embodiment of the invention discloses a region segmentation method and device for a hearing aid shell three-dimensional model and a medium, and relates to the technical field of three-dimensional digital modeling. The method comprises the following steps: acquiring a hearing aid shell three-dimensional model to be segmented and an external ear three-dimensional model of a source of the hearing aid shell three-dimensional model; performing region segmentation on the three-dimensional model of the external ear by using a three-dimensional segmentation algorithm based on deep learning to obtain an ear sample region to which each point in the three-dimensional model of the external ear belongs; performing spatial registration on the hearing aid shell three-dimensional model and the external ear three-dimensional model to enable the spatial positions of the two three-dimensional models to be most similar; and determining corresponding points of each point in the hearing aid shell three-dimensional model in the registered external ear three-dimensional model, and taking the ear sample area to which each corresponding point belongs as the ear sample area to which each point in the hearing aid shell three-dimensional model belongs. According to the embodiment of the invention, accurate region division of the hearing aid shell three-dimensional model can be automatically realized, and the production detection efficiency of the hearing aid shell is greatly improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the technical field of three-dimensional digital modeling, and in particular, to a method, device, and medium for region segmentation of a three-dimensional model of a hearing aid shell. Background Art

[0002] In the production of customized hearing aids (including the ear molds of in-the-ear hearing aids and behind-the-ear hearing aids), it is first necessary to obtain a three-dimensional model of the wearer's outer ear, as Figure 1 shown. Specifically, a rapid-curing material can be injected into the user's external auditory canal to the auricle part using a syringe. After it solidifies, the ear impression is taken out, and then the ear impression of the wearer is three-dimensionally scanned using a three-dimensional scanner to obtain a digital three-dimensional model of the outer ear, which is also called the original three-dimensional ear sample.

[0003] The original three-dimensional ear sample is the core key for subsequent hearing aid production. Production personnel need to use three-dimensional CAD software to perform a series of customized processing operations such as grinding, thickening, and cutting on different parts of the original three-dimensional ear sample, and finally obtain a three-dimensional model of the hearing aid shell (including the three-dimensional model of the in-the-ear hearing aid shell and the ear mold three-dimensional model of the behind-the-ear hearing aid), as Figure 2 shown, and perform 3D printing according to this three-dimensional model to complete the production of the hearing aid.

[0004] In the automated inspection process of hearing aid production, for different regional parts of the shell, the inspection standards are different. Therefore, it is necessary to perform ear sample region segmentation on the three-dimensional model of the hearing aid shell to clarify the corresponding outer ear parts of each part in the model, such as the first bend, the second bend, the tragus, etc.

[0005] In the prior art, the region segmentation of the three-dimensional model of the hearing aid shell is mostly based on visual observation, and manual visual determination is performed according to the biological definitions of different regions. The efficiency is not high, and the accuracy highly depends on the experience and theoretical knowledge level of the operator. In addition, after the three-dimensional model of the hearing aid shell has been edited and processed (ground, cut, thickened, etc.) in CAD software, it has already been greatly distorted compared to the original three-dimensional ear sample. It is difficult to directly perform region segmentation by visual observation, resulting in large errors in the segmentation results. If these errors extend to the wearing link, it will also cause differences between the problem descriptions of uncomfortable wearers and the actual situation, thus affecting the accuracy of production personnel in problem positioning and formulating rework strategies and reducing the user experience.

[0006] Therefore, there is an urgent need to propose a method for region segmentation of a three-dimensional model of a hearing aid shell, which can automatically and accurately identify and segment each part region of the hearing aid shell. Summary of the Invention

[0007] An embodiment of the present invention provides a method, device, and medium for region segmentation of a three-dimensional model of a hearing aid housing to solve the above technical problems.

[0008] In a first aspect, an embodiment of the present invention provides a method for region segmentation of a three-dimensional model of a hearing aid housing, including:

[0009] Obtain a three-dimensional model of the hearing aid housing to be segmented and the three-dimensional model of the outer ear from which it is derived;

[0010] Use a three-dimensional segmentation algorithm based on deep learning to perform region segmentation on the three-dimensional model of the outer ear to obtain the ear sample regions to which each point in the three-dimensional model of the outer ear belongs;

[0011] Perform spatial registration on the three-dimensional model of the hearing aid housing and the three-dimensional model of the outer ear to make the spatial positions of the two three-dimensional models most similar;

[0012] Determine the corresponding points of each point in the three-dimensional model of the hearing aid housing in the registered three-dimensional model of the outer ear, and use the ear sample regions to which the corresponding points belong as the ear sample regions to which each point in the three-dimensional model of the hearing aid housing belongs.

[0013] In a second aspect, an embodiment of the present invention provides an electronic device, which includes:

[0014] One or more processors;

[0015] A memory for storing one or more programs,

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the method for region segmentation of the three-dimensional model of the hearing aid housing according to any embodiment.

[0017] In a third aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the method for region segmentation of the three-dimensional model of the hearing aid housing according to any embodiment.

[0018] In summary, this embodiment provides a method, device, and medium for region segmentation of a three-dimensional model of a hearing aid shell, which can automatically achieve accurate region division of the three-dimensional model of the hearing aid shell. Specifically, after editing and processing operations such as grinding, cutting, and thickening, the three-dimensional model of the hearing aid shell has a large distortion compared to the original three-dimensional ear mold. It is difficult to directly perform region segmentation by visual inspection or deep learning. Therefore, in this embodiment, a deep learning method is first used to perform region segmentation on the three-dimensional model of the outer ear from which the hearing aid shell is derived, and using its rich and accurate structural information, it is divided into a variety of refined regions; then, spatial registration is performed on the three-dimensional model of the outer ear and the three-dimensional model of the hearing aid shell to make their spatial positions match; and then, using the matched spatial position points, the segmentation result of the three-dimensional model of the outer ear is extended to the three-dimensional model of the hearing aid shell to achieve accurate region division of the hearing aid shell. This method improves the accuracy of region segmentation, replaces the process of manual visual inspection for further subdivision, and greatly improves the production and detection efficiency of the hearing aid shell. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0020] Figure 1 is a schematic diagram of a three-dimensional model of an outer ear in the prior art;

[0021] Figure 2 is Figure 1 a schematic diagram of the corresponding three-dimensional model of the hearing aid shell;

[0022] Figure 3 is a flowchart of a method for region segmentation of a three-dimensional model of a hearing aid shell provided by an embodiment of the present invention;

[0023] Figure 4 is a schematic structural diagram of an end-to-end deep learning network provided by an embodiment of the present invention;

[0024] Figure 5 is a front schematic diagram of the region segmentation result of a three-dimensional model of an outer ear provided by an embodiment of the present invention;

[0025] Figure 6 is Figure 5 a reverse schematic diagram of the region segmentation result of the corresponding three-dimensional model of the outer ear;

[0026] Figure 7This is a registration schematic diagram of a three-dimensional model of the outer ear and a three-dimensional model of a hearing aid shell provided by an embodiment of the present invention;

[0027] Figure 8 is corresponding to Figure 5 、 Figure 6 front and back schematic diagrams of the segmentation result of the corresponding three-dimensional model area of the hearing aid shell;

[0028] Figure 9 This is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. Detailed implementation manners

[0029] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the scope protected by the present invention.

[0030] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus cannot be construed as a limitation of the present invention. In addition, the terms "first", "second", and "third" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance.

[0031] In the description of the present invention, it should also be noted that unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0032] Figure 3 This is a flowchart of a method for region segmentation of a three-dimensional model of a hearing aid shell provided by an embodiment of the present invention. This method is applicable to the situation of region segmentation of a three-dimensional model of an in-the-ear hearing aid shell or a three-dimensional model of an ear mold of a behind-the-ear hearing aid, and is executed by an electronic device. As Figure 3 shown, this method specifically includes:

[0033] S110. Obtain the three-dimensional model of the hearing aid shell to be segmented and the three-dimensional model of the outer ear from which it is derived.

[0034] As described above, for a certain user, after a series of processes such as polishing, thickening, and cutting on the three-dimensional model of the outer ear, the three-dimensional model of the hearing aid shell of this user can be obtained. In addition to obtaining this model as the object of region segmentation in this embodiment, the three-dimensional model of the outer ear of this user is also obtained and used as the data source of the entire method together.

[0035] S120. Use a three-dimensional segmentation algorithm based on deep learning to perform region segmentation on the three-dimensional model of the outer ear to obtain the ear sample regions to which each point in the three-dimensional model of the outer ear belongs.

[0036] From Figure 1 and Figure 2 it can be seen that the three-dimensional model of the hearing aid shell only covers a part of the outer ear, while the three-dimensional model of the outer ear has a wider coverage of the outer ear and richer structural information; at the same time, after a series of processes such as polishing, cutting, and thickening on the three-dimensional model of the hearing aid shell, the reflection of the ear structure itself has been greatly distorted, so the structural information of the three-dimensional model of the outer ear is more accurate.

[0037] Based on this, in this embodiment, the region segmentation is not directly performed on the three-dimensional model of the hearing aid shell, but the region segmentation is first performed on the three-dimensional model of the outer ear, and its rich and accurate structural information is used to improve the accuracy of region segmentation. Subsequently, the segmentation result is used as auxiliary information to complete the region segmentation of the three-dimensional model of the hearing aid shell.

[0038] Furthermore, in the usual ear sample region division, the original three-dimensional ear sample is usually divided into a bottom surface region, a bend region, a helix region, and a main body region. This division method no longer meets the miniaturization requirements of customized hearing aids (especially for in-the-ear hearing aids, and even more for ultra-small in-the-ear hearing aids). Therefore, this embodiment proposes a more detailed ear sample region division method, dividing the entire original three-dimensional ear sample into a first bend, a second bend, a tragus, an antitragus, an ear concha, an ear hook, and other regions. And the set of ear sample regions covered by the three-dimensional model of the hearing aid shell is a subset of the set of ear sample regions covered by the three-dimensional model of the outer ear, that is, the three-dimensional model of the hearing aid shell only covers several (less than 7) of the above 7 regions.

[0039] Optionally, an end-to-end deep learning network can be used to realize the automatic segmentation of the region of the three-dimensional model of the outer ear. Among them, the input of the deep learning network is the point cloud data information of the three-dimensional model of the outer ear, including the coordinates and normal vectors of each point in the point cloud; the output is the region category of each point in the point cloud, and the above 7 region types can be marked as 0-6 in sequence.

[0040] Exemplarily, assume that the point cloud data to be processed by the deep learning network includes m points, and the coordinates of the i-th (i = 1, 2,..., m) point are , then the point cloud data can be expressed as:

[0041]

[0042] For this point cloud data, this embodiment provides two types of deep learning network point cloud inputs:

[0043] The first type is the point cloud input P with normal vectors, and the format is:

[0044]

[0045] Among them, are the coordinates of the normal vector of the i-th point in the point cloud along the three directions of X, Y, and Z in the three-dimensional space. The three-dimensional model of the outer ear is a mesh model, and the normal vector of each point in the point cloud is the normal vector of the outer surface of the model.

[0046] The second type is the point cloud input including the above point information P and mesh information. Among them, the mesh information includes:

[0047] The adjacency matrix A of point to point = Among them, represents whether there is an adjacency relationship between the i-th point and the j-th point in the point cloud; if , then there is an adjacency relationship; if , then there is no adjacency relationship; and

[0048] The triangular patch matrix R = Among them, , and respectively represent the three points in the Q-th (Q = 1, 2,..., r) triangular patch in the three-dimensional model of the outer ear, where r represents the number of triangular patches in the three-dimensional model.

[0049] This embodiment uses the same backbone network structure for the above two types of point cloud inputs. It only needs to transform the input dimension to be consistent with the data dimension of the input point cloud through a convolutional layer at the front end of the backbone network structure. Of course, the richer the input data, the more accurate the final segmentation result. In practical applications, the point cloud input format and network structure can be flexibly selected according to the available data sources, the training computing power of the deep learning network, and the accuracy requirements, etc. This embodiment does not make specific restrictions. For the sake of simplicity of description, the following takes the first type of point cloud input as an example to illustrate the three-dimensional segmentation algorithm based on deep learning.

[0050] First, preprocess the three-dimensional model of the outer ear.

[0051] Specifically, first perform isotropic mesh reconstruction on the 3D data of the ear impression obtained by scanning to make the sizes of the triangular patches in the reconstructed model uniform; then denoise the reconstructed 3D data, for example, use the adaptive bilateral filtering algorithm to remove the noise points generated due to scanning errors or environmental interference. This method can process the overly sparse or overly dense points in the 3D data and further ensure the uniformity of the points.

[0052] After denoising, resample the 3D data to control the number of point cloud data within a certain range. Optionally, the farthest distance sampling can be performed. In this method, the distance between each sampling point and the previous sampling point is the largest, which can fully ensure the uniformity of sampling and the basic shape of the 3D model. Finally, the number of points can be controlled within 10,000. For the case where the number of points in the 3D data is large, the efficiency of subsequent training and inference of the deep learning network is better after sampling. The point cloud matrix output after sampling is denoted as .

[0053] According to the center point of the entire ear sample in the three-dimensional space after denoising, perform normalization on :

[0054]

[0055]

[0056]

[0057]

[0058] Among them, centerX, centerY, and centerZ respectively represent the coordinates of the center point of in the X, Y, and Z dimensions. max(X), max(Y), and max(Z) respectively represent the maximum values of the coordinates of all points in in the X, Y, and Z dimensions. min(X), min(Y), and min(Z) respectively represent the minimum values of the coordinates of all points in in the X, Y, and Z dimensions. represents the normalized point cloud data. It is worth mentioning that the center point calculated in this embodiment reflects the spatial center occupied by the entire ear sample. Therefore, the average of the coordinates of all points is not calculated because the average value obtained is affected by the density of the points and cannot accurately reflect the spatial center occupied by the entire ear sample.

[0059] After the preprocessing is completed, format the final 3D model into the input data of the first type and perform region segmentation on the end-to-end deep learning network at the input end. Figure 4 is a schematic structural diagram of this deep learning network. AsFigure 4 As shown, the network includes a point feature extraction network and a local area feature extraction network.

[0060] Among them, in the point feature extraction network, through layer-by-layer convolution operations, the input 3D point cloud data is converted into a representation in a high-dimensional feature space. Specifically, the formula for the convolution operation is:

[0061]

[0062] Among them, represents the spatial position information of the i-th point in the input point cloud, including the coordinates and normal vector of the point; represents the data corresponding to the point after the convolution operation, represents the weight vector, represents the kernel function of the convolution kernel, represents the kernel function centered on the i-th point represents the spatial position information of the points covered by the kernel function, and N represents the kernel function centered on the i-th point represents the number of points covered by the kernel function. After a series of convolutional layers, the data of each point is mapped to a high-dimensional feature vector , and this feature is an implicit feature containing the spatial information of the point, providing accurate point-level information for subsequent end-to-end per-point region segmentation. Optionally, the above-mentioned point feature extraction can be achieved through multi-step 1×1 convolution kernels. Specify the convolution kernel size as 1×1, the input dimension as 6, and the output dimension as the dimension of the high-dimensional feature.

[0063] The local area feature extraction network includes multiple layers of sparse convolutional layers, pooling layers, and fully connected layers. Among them, sparse convolution is the main body of the local area feature extraction network. In each convolutional layer, the local geometric features of the shell ladder surface are extracted through the sliding operation of a sparse convolution kernel (such as a 6×6×6 convolution kernel). The formula for each sparse convolution is as follows:

[0064]

[0065] Among them, represents the number of non-empty positions in the input point cloud, and respectively represent the weight and kernel function of the sparse convolution, represents the kernel function centered on the i-th point in the input point cloud represents the spatial position information of the non-empty positions covered by the kernel function except for the i-th point, represents the data corresponding to the point after the sparse convolution operation.

[0066] The pooling layer downsamples the features extracted by the coefficient convolution layer, reducing the amount of data while retaining key features. Optionally, the number of pooling layers can be less than that of the sparse convolution layers. Generally speaking, the more complex the shape of the ear mold, the fewer the number of pooling layers can be. The fully connected layer integrates the extracted features to obtain a vector that can represent the overall features of the 3D ear sample.

[0067] What the local region feature extraction network finally outputs is the feature vector of the local region where each point in the point cloud data is located, such as a 512-dimensional feature vector, which reflects the local geometric characteristics of the outer ear. It should be noted that the local region here refers to a spatial range around a certain point, not the 7 types of regions to be recognized, and they should be distinguished.

[0068] After the two feature extraction networks respectively complete feature extraction, for each point in the point cloud data, its point feature vector and the features of the local region where it is located are passed through an additional feature fusion matrix for weighted processing to obtain the final feature data of each point. Among them, can be understood as a matrix of the number of point clouds × 2, which is a learnable parameter matrix. The dimension 2 therein respectively corresponds to the weights of the point features and the local features, and in the training process, the model is made to learn whether each point in the point cloud depends more on the point information or the local information.

[0069] Input the final feature vector of each point into a multi-layer perceptron to output the probability that each point belongs to each of the 7 types of regions. The region type with the highest probability is the region type to which the point belongs. In this way, the positions of the 7 regions in the 3D model of the outer ear are obtained through the region segmentation algorithm, as shown in Figure 5 and Figure 6 where different colors represent different region types.

[0070] S130: Perform spatial registration on the 3D model of the hearing aid shell and the 3D model of the outer ear to make the spatial positions of the two 3D models most similar.

[0071] In this step, spatial registration is performed on the 3D model of the hearing aid shell to be partitioned and the segmented 3D model of the outer ear to make the spatial similarity of the two models the largest. To better illustrate this process, first, Figure 7 exemplarily shows a spatial position of the two models after registration. Among them, the registered 3D outer ear model is marked within the red frame, and the registered 3D model of the hearing aid shell is marked within the black frame. It can be seen that the ear sample area covered by the hearing aid shell is only a part of the ear sample area covered by the 3D model of the outer ear. Making the spatial positions and the trending shapes of the same ear sample areas in the two models closest means that the spatial registration is completed.

[0072] To achieve the above object, this embodiment provides a method for reinforcement learning. Using the features of the three-dimensional model of the hearing aid shell and the three-dimensional model of the outer ear as state variables, and using the spatial transformation from any one of the two three-dimensional models to the other three-dimensional model as an action variable, a reinforcement learning model is constructed. Through reinforcement learning, the spatial transformation of any one of the three-dimensional models is gradually performed. After each spatial transformation, with the increase in the spatial morphological similarity between the two three-dimensional models as the reward strategy, the action variable for the next spatial transformation is determined, and the spatial registration of the two three-dimensional models is gradually realized.

[0073] In a specific embodiment, the point cloud PO of the three-dimensional model of the outer ear (including m data points) can be expressed as:

[0074]

[0075] The point cloud PW of the three-dimensional model of the hearing aid shell (including n data points) can be expressed as:

[0076]

[0077] It is worth mentioning that if the three-dimensional model of the outer ear is preprocessed in S120, the three-dimensional model of the hearing aid shell in this step can be obtained by processing the three-dimensional model of the outer ear before preprocessing or by processing the three-dimensional model of the outer ear after preprocessing. This embodiment does not make specific restrictions.

[0078] Based on the above point cloud data, the state variables and action variables of the reinforcement learning model can be constructed.

[0079] Specifically, for the state variables, the two sets of point cloud data can be transformed into global feature vectors through feature embedding, and the state variables are generated using the two global feature vectors. Optionally, feature calculation is performed on the three-dimensional model PO of the outer ear to construct a high-dimensional feature fo; feature calculation is performed on the three-dimensional model PW of the hearing aid shell to construct a high-dimensional feature fw; then the state variable Status can be expressed as:

[0080] Status = concat(fo, fw)

[0081] Among them, concat(,) represents the concatenation operation. Optionally, the feature calculation here can reuse the point feature extraction network in S120, and a max pooling layer is added after the point feature extraction network. The maximum value of all points is selected in each feature dimension, and thus the high-dimensional feature of the entire point cloud data can be obtained. Exemplarily, the dimensions of fo and fw can be 1024.

[0082] At the same time, for the action variables, the spatial transformation from any one of the two three-dimensional models to the other three-dimensional model can be used as the action space, and the action space is divided to obtain each action variable.

[0083] Optionally, first perform direction division, dividing the three-dimensional space into three sets of directions Direct-Space = [X, Y, Z].

[0084] Then, according to the set unit translation distance, divide the translational motion space along a single direction into multiple discrete translational motions. Exemplarily, assuming that the minimum translation distance in the three-dimensional space is 0.05, then multiple discrete translation distances can be divided with this minimum translation distance as the unit, and the translational motion space Lim_T can be expressed as:

[0085] Lim_T = [-0.2, -0.1, -0.05, 0, 0.05, 0.1, 0.2].

[0086] Similarly, according to the set unit rotation angle, divide the rotational motion space along a single direction into multiple discrete rotation angles, and the rotational motion space Lim_R can be expressed as:

[0087] Lim_R = Lim_T × π / 2 = [-π / 10, -π / 20, -π / 40, 0, π / 40, π / 20, π / 10].

[0088] The above direction division, as well as translation and rotation division, can be simply expressed as the direction space [0, 1, 2] and the scale space [0, 1, 2, 3, 4, 5, 6, 7]. Each combination of direction, translation scale, and rotation scale constitutes an action variable.

[0089] Furthermore, according to each action variable, an action function for spatial transformation from one three-dimensional model to another three-dimensional model can be constructed, and this action function can be expressed as a combination of a translation function and a rotation function.

[0090] Among them, the translation function Translate(Direct, Lim) represents translation along a certain direction. The direction of translation is the direction represented by the Direct-th element in Direct-Space, and the translation distance is the distance corresponding to the Lim-th element in. The following will take the spatial transformation of PO as an example for illustration. Then the translated point cloud PO1’ = PO + Translate(Direct, Lim).

[0091] The rotation function Rotate(Direct, Lim) represents rotation along a certain direction. The direction of rotation is the direction represented by the Direct-th element in Direct-Space, and the rotation angle is The angle corresponding to the Lim-th element in. The rotated point cloud PO2’ = Rotate(Direct, Lim)PO.

[0092] It can be expressed in matrix form as:

[0093]

[0094]

[0095] After the state variables and action variables are constructed, it is also necessary to construct the reward function of the reinforcement learning model. In this embodiment, the Chamfer distance between two point cloud sets is used to characterize the spatial similarity between two 3D models. The calculation formula of the Chamfer distance is:

[0096]

[0097] Among them, U and V are two point cloud sets to be compared, and u and v represent the points in U and V respectively, represents the number of points in the point cloud set U. The smaller the Chamfer distance between two point cloud sets, the higher the similarity. Based on the Chamfer distance, the following reward function can be constructed:

[0098] Reward = CD(PO, PW) - CD(PO’, PW)

[0099] Among them, Reward represents the reward value, PO’ represents PO after spatial transformation, CD(PO, PW) represents the Chamfer distance between PO before spatial transformation and PW, and CD(PO’, PW) represents the Chamfer distance between PO after spatial transformation and PW. Under this reward function, if the Chamfer distance between two 3D models becomes closer through spatial transformation, a positive reward is given to the current action; if the Chamfer distance between two 3D models becomes farther through spatial transformation, a negative reward is given to the current action, so as to prompt the two 3D models to continuously approach in the direction of decreasing Chamfer distance.

[0100] Based on the above state variables, action variables and reward function, reinforcement learning training can be carried out. Among them, the reinforcement learning model includes an Actor network and a decision network. The feature extraction network that generates the above state variable Status can be used as the Actor network, and Status is the output of the Actor network; the decision network can adopt a multi-layer perceptron structure or a Linear-based transformer network structure, which is used to evaluate each action variable according to Status and the reward function, and determine the next action variable. The reward value of each time is accumulated during the training process:

[0101] Rewards = SUM(CD(PO, PW) - CD(Dot(Rotate[Lim], PO) + Translate[Lim], PW))

[0102] Among them, Dot represents the dot product of matrices. When the cumulative reward value Rewards no longer increases, it is determined that the two models are registered, that is, the effect shown in Figure 7 is achieved. The point cloud PWO after the registration of PO can be expressed as:

[0103]

[0104] Among them, and respectively represent the rotation matrix and the translation matrix of the spatial transformation at the e-th step.

[0105] S140. Determine the corresponding points of each point in the three-dimensional model of the hearing aid shell in the three-dimensional model of the outer ear after registration, and use the ear sample area to which each corresponding point belongs as the ear sample area to which each point in the three-dimensional model of the hearing aid shell belongs.

[0106] After registration, n points in the three-dimensional model of the hearing aid shell and m points in the three-dimensional model of the outer ear respectively correspond to a distance, forming the following distance matrix :

[0107]

[0108] Among them, respectively represent the distance between the j-th point in the three-dimensional model of the hearing aid shell and the i-th point in the three-dimensional model of the outer ear.

[0109] For each point j in the three-dimensional model of the hearing aid shell, determine , , … The minimum value among them, and the point in the three-dimensional model of the outer ear corresponding to this minimum value is the corresponding point of point j; the region type to which this corresponding point belongs is the region type to which point j belongs. Taking the three-dimensional model of the outer ear shown in Figure 5 , Figure 6 as an example, the segmentation result of the corresponding three-dimensional model of the hearing aid shell is as shown in Figure 8 . It can be seen that Figure 8 only includes the two-bend (red), one-bend (green), concha (blue) and tragus (yellow) regions, and does not include the antitragus, ear hook, other (such as the bottom surface) regions.

[0110] In summary, this embodiment provides a method for region segmentation of a three-dimensional model of a hearing aid shell, which can automatically achieve accurate region division of the three-dimensional model of the hearing aid shell. Specifically, after editing and processing operations such as grinding, cutting, and thickening, the three-dimensional model of the hearing aid shell has a large distortion relative to the original three-dimensional ear mold. It is difficult to directly perform region segmentation by visual inspection or deep learning. Therefore, in this embodiment, a deep learning method is first used to perform region segmentation on the three-dimensional model of the outer ear from which the hearing aid shell is derived. Using its rich and accurate structural information, it is divided into seven regions: the first bend, the second bend, the tragus, the antitragus, the concha, the earhook, and others. Then, spatial registration is performed on the three-dimensional model of the outer ear and the three-dimensional model of the hearing aid shell to make their spatial positions match. Then, using the matched spatial position points, the segmentation result of the three-dimensional model of the outer ear is extended to the three-dimensional model of the hearing aid shell to achieve accurate region division of the hearing aid shell. This method improves the accuracy of region segmentation, replaces the process of further subdividing the range by manual visual inspection, and greatly improves the production and detection efficiency of the hearing aid shell.

[0111] Figure 9 FIG. is a schematic structural diagram of an electronic device provided by an embodiment of the present invention, as Figure 9 shown. The device includes a processor 60, a memory 61, an input device 62, and an output device 63; the number of processors 60 in the device can be one or more, Figure 9 and one processor 60 is taken as an example here; the processor 60, the memory 61, the input device 62, and the output device 63 in the device can be connected through a bus or other means, Figure 9 and the connection through the bus is taken as an example here.

[0112] The memory 61, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as program instructions / modules corresponding to the method for region segmentation of the three-dimensional model of the hearing aid shell in the embodiment of the present invention. The processor 60 executes various functional applications and data processing of the device by running the software programs, instructions, and modules stored in the memory 61, that is, implements the above-mentioned method for region segmentation of the three-dimensional model of the hearing aid shell.

[0113] The memory 61 may mainly include a program storage area and a data storage area. Among them, the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 61 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some instances, the memory 61 may further include a memory remotely provided with respect to the processor 60, and these remote memories may be connected to the device through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0114] The input device 62 may be used to receive input digital or character information and generate key signal inputs related to user settings and function controls of the device. The output device 63 may include a display device such as a display screen.

[0115] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the region segmentation method of the three-dimensional model of the hearing aid housing in any embodiment.

[0116] The computer storage medium of the embodiment of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the computer-readable storage medium include: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer-readable storage medium may be any tangible medium that contains or stores a program, and this program may be used by or in combination with an instruction execution system, apparatus, or device.

[0117] The computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries the computer-readable program code. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable medium other than the computer-readable storage medium, and this computer-readable medium may send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device.

[0118] The program code contained on a computer-readable medium can be transmitted with any suitable medium, including but not limited to wireless, wire, optical fiber cable, RF, etc., or any suitable combination of the above.

[0119] The computer program code for performing the operations of the present invention can be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as C language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0120] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some or all of the technical features. These modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the technical solutions of the embodiments of the present invention.

Claims

1. A method for region segmentation of a three-dimensional model of a hearing aid shell, characterized in that, Including: Obtain the three-dimensional model of the hearing aid shell to be segmented and the three-dimensional model of the outer ear from which it is derived; Use a three-dimensional segmentation algorithm based on deep learning to perform regional segmentation on the three-dimensional model of the outer ear to obtain the ear sample regions to which each point in the three-dimensional model of the outer ear belongs; Perform spatial registration on the three-dimensional model of the hearing aid shell and the three-dimensional model of the outer ear to make the spatial positions of the two three-dimensional models most similar; Determine the corresponding points of each point in the three-dimensional model of the hearing aid shell in the registered three-dimensional model of the outer ear, and use the ear sample regions to which the corresponding points belong as the ear sample regions to which each point in the three-dimensional model of the hearing aid shell belongs.

2. The method according to claim 1, wherein The three-dimensional model of the hearing aid shell is obtained by processing the three-dimensional model of the outer ear; The set of ear sample regions covered by the three-dimensional model of the hearing aid shell is a subset of the set of ear sample regions covered by the three-dimensional model of the outer ear.

3. The method according to claim 1, characterized in that The using a three-dimensional segmentation algorithm based on deep learning to perform regional segmentation on the three-dimensional model of the outer ear to obtain the ear sample regions to which each point in the three-dimensional model of the outer ear belongs includes: Use a point feature extraction network based on deep learning to convert the spatial position information of each point in the three-dimensional model of the outer ear into high-dimensional features of each point; Use a local region feature extraction network based on deep learning to extract the geometric features of local regions from the three-dimensional model of the outer ear to obtain the features of the local regions to which each point belongs; Use a feature fusion matrix to fuse and classify the high-dimensional features of each point and the features of the local regions to which they belong to obtain the ear sample regions to which each point belongs.

4. The method according to claim 1, characterized in that, The performing spatial registration on the three-dimensional model of the hearing aid shell and the three-dimensional model of the outer ear to make the spatial positions of the two three-dimensional models most similar includes: Construct a reinforcement learning model with the high-dimensional features of the three-dimensional model of the hearing aid shell and the three-dimensional model of the outer ear as state variables and the spatial transformation from any one of the two three-dimensional models to the other as action variables; Based on the reinforcement learning model, gradually perform spatial transformation on the any one of the three-dimensional models, and after each spatial transformation, use the increase in the spatial position similarity of the two three-dimensional models as the reward strategy to decide the action variable of the next spatial transformation, and gradually achieve the spatial registration of the two three-dimensional models.

5. The method according to claim 4, characterized in that The using the high-dimensional features of the three-dimensional model of the hearing aid shell and the three-dimensional model of the outer ear as state variables includes: Extract the high-dimensional features of each point in the three-dimensional model of the hearing aid shell and the three-dimensional model of the outer ear from a three-dimensional segmentation algorithm based on deep learning; Perform pooling operations on the high-dimensional features of each point in the two three-dimensional models respectively to obtain the high-dimensional features of the two three-dimensional models respectively.

6. The method according to claim 4, wherein The using the spatial transformation from any one of the two three-dimensional models to the other as action variables includes: According to the set unit translation distance and unit rotation angle, divide the translation transformation and rotation transformation along a single direction into multiple discrete actions respectively; According to each discrete action, construct an action function for the spatial transformation from any one of the two three-dimensional models to the other.

7. The method according to claim 4, wherein The using the increase in the spatial position similarity of the two three-dimensional models as the reward strategy to decide the action variable of the next spatial transformation and gradually achieve the spatial registration of the two three-dimensional models includes: Taking the Chamfer distance difference between the two 3D models before and after each spatial transformation as the reward function, the action variable of the next spatial transformation is determined. With the maximum cumulative reward value as the optimization goal, the spatial registration of the two 3D models is gradually realized.

8. The method according to claim 1, wherein The determination of the corresponding points of each point in the 3D model of the hearing aid shell in the 3D model of the outer ear after registration includes: In the 3D model of the outer ear after registration, the corresponding points closest to each point in the 3D model of the hearing aid shell are determined.

9. An electronic device, characterized in that, Including: One or more processors; A memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method for region segmentation of the 3D model of the hearing aid shell according to any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, Stored thereon is a computer program which, when executed by a processor, implements the method for region segmentation of the 3D model of the hearing aid shell according to any one of claims 1-8.

Images