Method, device and medium for regional segmentation of a three-dimensional model of a hearing aid shell

Abstract

The embodiment of the present invention discloses a method, device and medium for regional segmentation of a three-dimensional model of a hearing aid shell, and relates to the field of three-dimensional digital modeling technology. The method includes: obtaining a three-dimensional model of a hearing aid shell to be segmented, and a three-dimensional model of the outer ear from which it is derived; performing regional segmentation on the three-dimensional model of the outer ear using a three-dimensional segmentation algorithm based on deep learning, and obtaining the ear-sample area to which each point in the three-dimensional model of the outer ear belongs; spatially aligning the three-dimensional model of the hearing aid shell with the three-dimensional model of the outer ear, so that the spatial positions of the two three-dimensional models are most similar; determining the corresponding points of each point in the three-dimensional model of the hearing aid shell in the aligned three-dimensional model of the outer ear, and using the ear-sample area to which each corresponding point belongs as the ear-sample area to which each point in the three-dimensional model of the hearing aid shell belongs. This embodiment can automatically realize the precise regional division of the three-dimensional model of the hearing aid shell, greatly improving the production and inspection efficiency of the hearing aid shell.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of three-dimensional digital modeling, and in particular to a method, device, and medium for segmenting a three-dimensional model of a hearing aid housing. Background Art

[0002] In the production of custom hearing aids (including ear molds for in-the-canal hearing aids and behind-the-ear hearing aids), it is first necessary to obtain a three-dimensional model of the wearer's outer ear, such as Figure 1 Specifically, a syringe can be used to inject the rapid-curing material into the user's external auditory canal and auricle. After solidification, the ear impression is taken out, and then a 3D scanner is used to perform a 3D scan of the wearer's ear impression to obtain a digital 3D model of the outer ear, which is also called an original 3D ear sample.

[0003] The original 3D ear sample is the key to the subsequent hearing aid production. The production staff needs to use 3D CAD software to perform a series of customized processing on different parts of the original 3D ear sample, such as grinding, thickening, cutting, etc., and finally obtain the 3D model of the hearing aid shell (including the 3D model of the ear canal hearing aid shell and the 3D model of the behind-the-ear hearing aid ear mold). Figure 2 As shown, 3D printing is performed based on the three-dimensional model to complete the hearing aid production.

[0004] During the automated inspection process of hearing aid manufacturing, different inspection standards apply to different areas of the shell. Therefore, the three-dimensional model of the hearing aid shell needs to be segmented into ear-like areas to clearly identify the external ear parts corresponding to each part of the model, such as the first bend, the second bend, the tragus, etc.

[0005] In existing technology, segmentation of 3D hearing aid housing models is often based on visual observation, with manual visual assessment based on biological definitions of different regions. This is inefficient and its accuracy is highly dependent on the operator's experience and theoretical knowledge. Furthermore, since 3D hearing aid housing models undergo significant editing and processing (such as polishing, cutting, and thickening) in CAD software, they are already significantly distorted compared to the original 3D ear sample. Direct visual segmentation is challenging, leading to significant errors in the segmentation results. These errors, if extended to the wearer's experience, can lead to discrepancies between the wearer's description of the discomfort and the actual situation, compromising the accuracy of problem identification and rework strategies developed by production personnel, and thus reducing the user experience.

[0006] Therefore, there is an urgent need to propose a region segmentation method for the three-dimensional model of the hearing aid shell, which can automatically and accurately identify and segment the various parts of the hearing aid shell. Summary of the Invention

[0007] Embodiments of the present invention provide a method, device, and medium for segmenting a region of a three-dimensional model of a hearing aid housing to solve the above-mentioned technical problems.

[0008] In a first aspect, an embodiment of the present invention provides a method for segmenting a region of a three-dimensional model of a hearing aid housing, comprising:

[0009] Obtaining a three-dimensional model of the hearing aid shell to be segmented and a three-dimensional model of the outer ear from which it is derived;

[0010] Performing regional segmentation on the three-dimensional outer ear model using a three-dimensional segmentation algorithm based on deep learning to obtain the ear-sample region to which each point in the three-dimensional outer ear model belongs;

[0011] spatially registering the hearing aid housing three-dimensional model with the outer ear three-dimensional model so that the spatial positions of the two three-dimensional models are most similar;

[0012] Corresponding points of each point in the three-dimensional model of the hearing aid housing in the registered three-dimensional model of the outer ear are determined, and the ear sample areas to which each corresponding point belongs are used as the ear sample areas to which each point in the three-dimensional model of the hearing aid housing belongs.

[0013] In a second aspect, an embodiment of the present invention provides an electronic device, comprising:

[0014] one or more processors;

[0015] a memory for storing one or more programs,

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the region segmentation method of the three-dimensional model of the hearing aid housing according to any one of the embodiments.

[0017] In a third aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the region segmentation method of the three-dimensional model of the hearing aid housing described in any embodiment.

[0018] In summary, this embodiment provides a method, device, and medium for segmenting a three-dimensional model of a hearing aid housing, which can automatically achieve accurate regional division of the three-dimensional model of the hearing aid housing. Specifically, because the three-dimensional model of the hearing aid housing has undergone significant distortion relative to the original three-dimensional ear sample after editing and processing operations such as grinding, cutting, and thickening, it is difficult to perform regional segmentation directly by visual observation or deep learning. Therefore, this embodiment first uses a deep learning method to segment the three-dimensional model of the outer ear, which is the source of the hearing aid housing, and uses its rich and accurate structural information to divide it into multiple refined regions. The three-dimensional model of the outer ear and the three-dimensional model of the hearing aid housing are then spatially aligned to match their spatial positions. The segmentation results of the three-dimensional model of the outer ear are then extended to the three-dimensional model of the hearing aid housing using the matching spatial position points, achieving accurate regional division of the hearing aid housing. This method improves the accuracy of regional segmentation, replaces the process of further subdividing the range by the human eye, and greatly improves the production and inspection efficiency of hearing aid housings. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0020] Figure 1 It is a schematic diagram of a three-dimensional model of the outer ear in the prior art;

[0021] Figure 2 is with Figure 1 a schematic diagram of the corresponding three-dimensional model of the hearing aid housing;

[0022] Figure 3 This is a flow chart of a method for segmenting a three-dimensional model of a hearing aid housing provided by an embodiment of the present invention;

[0023] Figure 4 1 is a schematic diagram of the structure of an end-to-end deep learning network provided by an embodiment of the present invention;

[0024] Figure 5 1 is a front view schematic diagram of a segmentation result of a three-dimensional outer ear model region provided by an embodiment of the present invention;

[0025] Figure 6 is with Figure 5 The corresponding reverse schematic diagram of the outer ear 3D model region segmentation result;

[0026] Figure 7is a schematic diagram of registration of a three-dimensional model of an outer ear and a three-dimensional model of a hearing aid housing provided by an embodiment of the present invention;

[0027] Figure 8 is with Figure 5 、 Figure 6 Schematic diagram of the front and back of the corresponding hearing aid shell 3D model area segmentation results;

[0028] Figure 9 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0029] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention are described clearly and completely below. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are also within the scope of protection of the present invention.

[0030] In the description of the present invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limitations on the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0031] In the description of the present invention, it should also be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood broadly. For example, they may refer to fixed, detachable, or integral connections; mechanical or electrical connections; direct or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0032] Figure 3 This is a flow chart of a method for regional segmentation of a hearing aid housing 3D model provided by an embodiment of the present invention. This method is applicable to the case of regional segmentation of a 3D model of an ear canal hearing aid housing or a 3D model of a behind-the-ear hearing aid ear mold, and is executed by an electronic device. Figure 3 As shown, the method specifically includes:

[0033] S110 , obtaining a three-dimensional model of the hearing aid shell to be segmented and a three-dimensional model of the outer ear from which it is derived.

[0034] As described above, for a particular user, after a series of processes such as polishing, thickening, and cutting are performed on the 3D outer ear model of the user, a 3D model of the user's hearing aid shell can be obtained. In addition to obtaining this model as the object for region segmentation, this embodiment also obtains the 3D outer ear model of the user, which together serve as the data source for the entire method.

[0035] S120. Perform regional segmentation on the three-dimensional outer ear model using a three-dimensional segmentation algorithm based on deep learning to obtain the ear-sample region to which each point in the three-dimensional outer ear model belongs.

[0036] Depend on Figure 1 and Figure 2 It can be seen that the three-dimensional model of the hearing aid shell only covers part of the outer ear, while the three-dimensional model of the outer ear covers a wider range of the outer ear and has richer structural information; at the same time, after a series of processing such as polishing, cutting and thickening, the three-dimensional model of the hearing aid shell itself has been greatly distorted in its reflection of the ear structure, so the structural information of the three-dimensional model of the outer ear is more accurate.

[0037] Based on this, this embodiment does not directly perform regional segmentation on the three-dimensional model of the hearing aid shell. Instead, it first performs regional segmentation on the three-dimensional model of the outer ear, using its rich and accurate structural information to improve the accuracy of regional segmentation. The segmentation result is then used as auxiliary information to complete the regional segmentation of the three-dimensional model of the hearing aid shell.

[0038] Furthermore, conventional ear sample region division typically divides the original 3D ear sample into the base, bend, helix, and main body regions. This division method no longer meets the miniaturization requirements of custom hearing aids (especially deep-canal hearing aids, and even ultra-small deep-canal hearing aids). Therefore, this embodiment proposes a more detailed ear sample region division method, dividing the entire original 3D ear sample into the first bend, the second bend, the tragus, the anti-tragus, the cavum concha, the ear hook, and other regions. The set of ear sample regions covered by the 3D hearing aid housing model is a subset of the set of ear sample regions covered by the 3D outer ear model. That is, the 3D hearing aid housing model only covers a few (less than 7) of the aforementioned seven regions.

[0039] Alternatively, an end-to-end deep learning network can be used to automatically segment the outer ear 3D model. The input of the deep learning network is the point cloud data of the outer ear 3D model, including the coordinates and normal vector of each point in the point cloud; the output is the region category of each point in the point cloud. The seven region types mentioned above can be labeled in order from 0 to 6.

[0040] For example, assume that the point cloud data to be processed by the deep learning network includes m points, and the coordinates of the i-th (i=1,2,…,m) point are , then the point cloud data can be expressed as:

[0041]

[0042] For this point cloud data, this embodiment provides two types of deep learning network point cloud input:

[0043] The first type is point cloud input P with normal vectors, the format is:

[0044]

[0045] in, is the coordinate of the normal vector of the i-th point in the point cloud along the X, Y, and Z directions in 3D space. The 3D model of the external ear is a mesh model, and the normal vector of each point in the point cloud is the normal vector of the outer surface of the model.

[0046] The second type includes the point cloud input of the above-mentioned point information P and mesh information, where the mesh information includes:

[0047] The point-to-point adjacency matrix A= ,in, Indicates whether there is an adjacency relationship between the i-th point and the j-th point in the point cloud; if , then there is an adjacency relationship; if , then there is no adjacency relationship; and

[0048] Triangle patch matrix R= ,in, 、 and They represent three points in the Qth (Q=1,2,…,r) triangle in the three-dimensional model of the outer ear, where r represents the number of triangles in the three-dimensional model.

[0049] This embodiment uses the same backbone network structure for the above two types of point cloud inputs. It is only necessary to transform the input dimension to be consistent with the data dimension of the input point cloud through the convolution layer at the front end of the backbone network structure. Of course, the richer the input data, the more accurate the final segmentation result. In practical applications, the point cloud input format and network structure can be flexibly selected based on the data source already mastered, the training computing power of the deep learning network, and the accuracy requirements. This embodiment does not impose specific restrictions. To simplify the description, the following uses the first type of point cloud input as an example to illustrate the three-dimensional segmentation algorithm based on deep learning.

[0050] First, the external ear 3D model is preprocessed.

[0051] Specifically, the 3D data of the scanned ear impression is first reconstructed into an isotropic mesh so that the size of the triangular facets in the reconstructed model is uniform; then the reconstructed 3D data is denoised, for example using an adaptive bilateral filtering algorithm to remove noise points caused by scanning errors or environmental interference. This method can process points that are too sparse or too dense in the 3D data, further ensuring the uniformity of the points.

[0052] After denoising, the 3D data is resampled to control the number of point cloud data within a certain range. Optionally, the longest distance sampling can be performed. In this method, the distance between each sampling point and the previous sampling point is the largest, which can fully ensure the uniformity of sampling and the basic form of the 3D model. Ultimately, the number of points can be controlled within 10,000. For cases where the number of points in the 3D data is large, it is more efficient to perform subsequent deep learning network training and inference after sampling. The point cloud matrix output after sampling is recorded as .

[0053] According to the center point of the entire ear sample in three-dimensional space after denoising, Perform normalization:

[0054]

[0055]

[0056]

[0057]

[0058] Among them, centerX, centerY and centerZ represent The coordinates of the center point in the X, Y and Z dimensions, max(X), max(Y) and max(Z) represent The maximum value of the coordinates of all points in the X, Y and Z dimensions, min(X), min(Y) and min(Z) respectively represent The minimum value of the coordinates of all points in the X, Y and Z dimensions, It is worth mentioning that the center point calculated in this embodiment reflects the spatial center occupied by the entire ear sample. Therefore, the coordinates of all points are not averaged. Because the average is affected by the density of the points, the average value obtained cannot accurately reflect the spatial center occupied by the entire ear sample.

[0059] After preprocessing, the final 3D model is formatted as the first type of input data and input into the end-to-end deep learning network for region segmentation. Figure 4 This is a structural diagram of the deep learning network, such as Figure 4 As shown, the network includes a point feature extraction network and a local area feature extraction network.

[0060] Among them, the point feature extraction network converts the input 3D point cloud data into a representation in a high-dimensional feature space through layer-by-layer convolution operations. Specifically, the formula for the convolution operation is:

[0061]

[0062] in, Represents the spatial position information of the i-th point in the input point cloud, including the coordinates and normal vector of the point; Represents the data corresponding to the point after the convolution operation, represents the weight vector, represents the kernel function of the convolution kernel, Represents the kernel function centered at the i-th point The spatial position information of the covered points, N represents the kernel function centered on the i-th point The number of points covered. After a series of convolutional layers, the data of each point are mapped into a high-dimensional feature vector This feature is an implicit feature that contains the spatial information of the point, providing accurate information of the point granularity for subsequent end-to-end point-by-point region segmentation. Optionally, the above point features can be extracted using a multi-step 1×1 convolution kernel, specifying a convolution kernel size of 1×1, an input dimension of 6, and an output dimension equal to the dimension of the high-dimensional feature.

[0063] The local region feature extraction network consists of multiple layers of sparse convolutional layers, pooling layers, and fully connected layers. Sparse convolution is the core of the local region feature extraction network. In each convolutional layer, a sparse convolution kernel (e.g., a 6×6×6 convolution kernel) is used to slide the local geometric features of the shell trapezoidal surface. The formula for each sparse convolution is as follows:

[0064]

[0065] in, represents the number of non-empty locations in the input point cloud, and Represent the weight and kernel function of sparse convolution respectively, Represents the kernel function centered at the i-th point in the input point cloud The spatial location information except for the non-empty locations covered, Represents the data corresponding to the point after the sparse convolution operation.

[0066] The pooling layer downsamples the features extracted by the coefficient convolution layer, reducing the amount of data while retaining key features. Optionally, the number of pooling layers can be fewer than the number of sparse convolution layers. Generally speaking, the more complex the ear mold shape, the fewer pooling layers are needed. The fully connected layer integrates the extracted features to obtain a vector that represents the overall characteristics of the 3D ear sample.

[0067] The local region feature extraction network ultimately outputs a feature vector for the local region of each point in the point cloud data, such as a 512-dimensional feature vector. This vector reflects the local geometric characteristics of the outer ear. It should be noted that the local region here refers to the spatial range surrounding a point, not the seven region types to be identified, and should be distinguished accordingly.

[0068] After the two feature extraction networks complete the feature extraction, for each point in the point cloud data, the feature vector of the point and the features of the local area are combined through an additional feature fusion matrix Perform weighted processing to obtain the final feature data of each point. It can be understood as a matrix of the number of point clouds × 2, which is a learnable parameter matrix. The dimension 2 corresponds to the weights of point features and local features respectively. During the training process, the model learns whether each point in the point cloud relies more on point information or local information.

[0069] The final feature vector of each point is input into the multi-layer perceptron, which outputs the probability that each point belongs to each of the seven region types. The region type with the highest probability is the region type to which the point belongs. In this way, the positions of the seven regions in the three-dimensional model of the outer ear are obtained through the region segmentation algorithm, as shown in the figure below. Figure 5 and Figure 6 As shown, different colors represent different area types.

[0070] S130: spatially align the hearing aid housing three-dimensional model with the outer ear three-dimensional model so that the spatial positions of the two three-dimensional models are most similar.

[0071] This step performs spatial registration of the partitioned hearing aid shell 3D model and the segmented outer ear 3D model to maximize the spatial similarity of the two models. Figure 7 This example illustrates the spatial positions of the two models after registration. The red frame shows the registered 3D outer ear model, while the black frame shows the registered 3D hearing aid housing model. As can be seen, the ear-like area covered by the hearing aid housing is only a portion of the area covered by the 3D outer ear model. This aligns the spatial positions and orientations of the same ear-like areas in the two models, effectively completing the registration.

[0072] To achieve the above objectives, this embodiment provides a reinforcement learning method, which uses the characteristics of the hearing aid shell three-dimensional model and the outer ear three-dimensional model as state variables, and uses the spatial transformation from any one of the two three-dimensional models to the other three-dimensional model as an action variable to construct a reinforcement learning model; through reinforcement learning, the any one of the three-dimensional models is gradually spatially transformed, and after each step of spatial transformation, the action variable for the next spatial transformation is determined based on the increasing spatial morphological similarity of the two three-dimensional models as a reward strategy, thereby gradually achieving spatial alignment of the two three-dimensional models.

[0073] In a specific embodiment, the point cloud PO of the outer ear three-dimensional model (including m data points) can be expressed as:

[0074]

[0075] The point cloud PW (including n data points) of the hearing aid shell 3D model can be expressed as:

[0076]

[0077] It is worth mentioning that if the outer ear three-dimensional model is preprocessed in S120, the hearing aid shell three-dimensional model in this step can be obtained by processing the outer ear three-dimensional model before preprocessing, or can be obtained by processing the outer ear three-dimensional model after preprocessing, and this embodiment does not impose specific restrictions.

[0078] Based on the above point cloud data, the state variables and action variables of the reinforcement learning model can be constructed.

[0079] Specifically, for state variables, feature embedding can be used to convert the two sets of point cloud data into global feature vectors, and the two global feature vectors can be used to generate state variables. Optionally, feature calculation can be performed on the outer ear 3D model PO to construct a high-dimensional feature fo; feature calculation can be performed on the hearing aid shell 3D model PW to construct a high-dimensional feature fw; then the state variable Status can be expressed as:

[0080] Status=concat(fo,fw)

[0081] Here, concat(,) represents a concatenation operation. Optionally, the feature calculation here can reuse the point feature extraction network in S120. A max pooling layer is added after the point feature extraction network to select the maximum value of all points in each feature dimension, thereby obtaining high-dimensional features for the entire point cloud data. For example, the dimensions of fo and fw can be 1024.

[0082] At the same time, for action variables, the spatial transformation from any one of the two three-dimensional models to the other three-dimensional model can be used as the action space, and the action space can be divided to obtain various action variables.

[0083] Optionally, first perform direction division to divide the three-dimensional space into a set of three directions Direct-Space=[X,Y,Z].

[0084] Then, based on the set unit translation distance, the translation action space along a single direction is divided into multiple discrete translation actions. For example, assuming that the minimum translation distance in the three-dimensional space is 0.05, multiple discrete translation distances can be divided based on the minimum translation distance. The translation action space Lim_T can be expressed as:

[0085] Lim_T=[-0.2, -0.1, -0.05, 0, 0.05, 0.1, 0.2].

[0086] Similarly, according to the set unit rotation angle, the rotation action space along a single direction is divided into multiple discrete rotation angles, and the rotation action space Lim_R can be expressed as:

[0087] Lim_R=Lim_T×π / 2=[-π / 10, -π / 20, -π / 40, 0, π / 40, π / 20, π / 10].

[0088] The above direction division, as well as the translation and rotation division can be simply expressed as the direction space [0, 1, 2] and the scale space [0, 1, 2, 3, 4, 5, 6, 7]. Each combination of direction, translation scale and rotation scale constitutes an action variable.

[0089] Furthermore, based on each action variable, an action function for spatial transformation from one of the two three-dimensional models to the other three-dimensional model can be constructed. The action function can be expressed as a combination of a translation function and a rotation function.

[0090] Among them, the translation function Translate(Direct, Lim) represents translation along a certain direction. The direction of translation is the direction represented by the Direct element in Direct-Space, and the translation distance is The following explanation takes the spatial transformation of PO as an example, and the translated point cloud PO1'=PO+Translate(Direct, Lim).

[0091] The rotation function Rotate(Direct, Lim) represents rotation along a certain direction. The direction of rotation is the direction represented by the Direct element in Direct-Space, and the angle of rotation is The angle corresponding to the Limth element in . The rotated point cloud PO2'=Rotate(Direct,Lim)PO.

[0092] It can be expressed in matrix form as:

[0093]

[0094]

[0095] After the state variables and action variables are constructed, the reward function of the reinforcement learning model needs to be constructed. In this embodiment, the Chamfer distance between two point cloud sets is used to represent the spatial similarity between two 3D models. The calculation formula of the Chamfer distance is:

[0096]

[0097] Among them, U and V are the two point cloud sets to be compared, u and v represent the points in U and V respectively. Represents the number of points in the point cloud set U. The smaller the Chamfer distance between two point cloud sets, the higher the similarity. Based on the Chamfer distance, the following reward function can be constructed:

[0098] Reward = CD(PO,PW)-CD(PO',PW)

[0099] Where Reward represents the reward value, PO' represents the PO after spatial transformation, CD(PO, PW) represents the Chamfer distance between PO and PW before spatial transformation, and CD(PO', PW) represents the Chamfer distance between PO and PW after spatial transformation. Under this reward function, if the spatial transformation shortens the Chamfer distance between the two 3D models, the current action is rewarded positively; if the spatial transformation shortens the Chamfer distance between the two 3D models, the current action is rewarded negatively, thereby encouraging the two 3D models to move closer together in the direction of decreasing Chamfer distance.

[0100] Based on the above state variables, action variables, and reward functions, reinforcement learning training can be performed. The reinforcement learning model includes an actor network and a decision network. The feature extraction network that generates the above state variable Status can be used as the actor network, and Status is the output of the actor network. The decision network can use a multi-layer perceptron structure or a linear-based transformer network structure to evaluate each action variable based on Status and reward functions and determine the next action variable. During training, the reward value is accumulated each time:

[0101] Rewards=SUM(CD(PO,PW)-CD(Dot(Rotate[Lim],PO)+Translate[Lim],PW))

[0102] Where Dot represents the dot product of the matrix. When the cumulative reward value Rewards no longer increases, it is determined that the two models have been aligned, that is, Figure 7 The point cloud PWO after PO registration can be expressed as:

[0103]

[0104] in, and They represent the rotation matrix and translation matrix of the e-th step spatial transformation respectively.

[0105] S140: Determine corresponding points of each point in the three-dimensional model of the hearing aid housing in the registered three-dimensional model of the outer ear, and use the ear sample areas to which each corresponding point belongs as the ear sample areas to which each point in the three-dimensional model of the hearing aid housing belongs.

[0106] After registration, each of the n points in the 3D model of the hearing aid shell and the m points in the 3D model of the outer ear corresponds to a distance, forming the following distance matrix: :

[0107]

[0108] in, They represent the distance between the jth point in the 3D model of the hearing aid shell and the i-th point in the 3D model of the outer ear.

[0109] For each point j in the 3D model of the hearing aid housing, determine 、 、… The minimum value in the 3D model of the outer ear corresponds to the point in the 3D model of the outer ear, which is the corresponding point of point j. The region type to which the corresponding point belongs is the region type to which point j belongs. Figure 5 、 Figure 6 Taking the outer ear 3D model shown in the figure as an example, the segmentation results of the corresponding hearing aid shell 3D model are as follows Figure 8 As shown, it can be seen that Figure 8 The image only includes the second bend (red), the first bend (green), the concha cavity (blue) and the tragus (yellow), but does not include the antitragus, ear hook and other areas (such as the bottom surface).

[0110] In summary, this embodiment provides a method for segmenting a 3D hearing aid housing model, capable of automatically and accurately segmenting the 3D hearing aid housing model. Specifically, because the 3D hearing aid housing model is significantly distorted relative to the original 3D ear sample after processing operations such as grinding, cutting, and thickening, direct segmentation via visual observation or deep learning is difficult. Therefore, this embodiment first employs a deep learning method to segment the 3D outer ear model, which serves as the source of the hearing aid housing. Leveraging its rich and accurate structural information, the 3D model is divided into seven regions: the first bend, the second bend, the tragus, the antitragus, the cavum concha, the ear hook, and other regions. The 3D outer ear model and the 3D hearing aid housing model are then spatially registered to match their spatial positions. Using the matched spatial positions, the segmentation results from the 3D outer ear model are then extended to the 3D hearing aid housing model, achieving precise segmentation of the hearing aid housing. This method improves the accuracy of segmentation, replaces the process of further segmentation by the naked eye, and significantly enhances the production inspection efficiency of hearing aid housings.

[0111] Figure 9 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention is shown in FIG. Figure 9 As shown, the device includes a processor 60, a memory 61, an input device 62 and an output device 63; the number of processors 60 in the device can be one or more. Figure 9 In the embodiment, a processor 60 is used as an example; the processor 60, the memory 61, the input device 62 and the output device 63 in the device can be connected by a bus or other means. Figure 9 The bus connection is taken as an example.

[0112] Memory 61, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the method for segmenting a three-dimensional hearing aid housing model in the embodiments of the present invention. Processor 60 executes the software programs, instructions, and modules stored in memory 61 to perform various functional applications and data processing of the device, thereby implementing the method for segmenting a three-dimensional hearing aid housing model.

[0113] The memory 61 may primarily include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function; the data storage area may store data generated based on the use of the terminal. Furthermore, the memory 61 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state memory device. In some instances, the memory 61 may further include memory remotely located relative to the processor 60, and these remote memories may be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0114] The input device 62 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the device. The output device 63 may include a display device such as a display screen.

[0115] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the method for segmenting a region of a three-dimensional model of a hearing aid housing according to any embodiment is implemented.

[0116] The computer storage medium of the embodiments of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or device.

[0117] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0118] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0119] Computer program code for performing the operations of the present invention can be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as C or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0120] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the technical solutions of the embodiments of the present invention.

Claims

1. A method for segmenting a three-dimensional model of a hearing aid housing, characterized in that: include: Obtaining a three-dimensional model of the hearing aid shell to be segmented and a three-dimensional model of the outer ear from which it is derived; Performing regional segmentation on the three-dimensional outer ear model using a three-dimensional segmentation algorithm based on deep learning to obtain the ear-sample region to which each point in the three-dimensional outer ear model belongs; Constructing a reinforcement learning model using the high-dimensional features of the hearing aid housing model and the outer ear model as state variables and the spatial transformation from one of the two three-dimensional models to the other as an action variable; Based on the reinforcement learning model, the spatial transformation of any three-dimensional model is gradually performed. After each spatial transformation, the action variable of the next spatial transformation is determined based on the increasing similarity of the spatial positions of the two three-dimensional models as a reward strategy, thereby gradually achieving spatial alignment of the two three-dimensional models. Corresponding points of each point in the three-dimensional model of the hearing aid housing in the registered three-dimensional model of the outer ear are determined, and the ear sample areas to which each corresponding point belongs are used as the ear sample areas to which each point in the three-dimensional model of the hearing aid housing belongs.

2. The method according to claim 1, characterized in that The three-dimensional model of the hearing aid housing is obtained by processing the three-dimensional model of the outer ear; The set of ear-like areas covered by the three-dimensional model of the hearing aid housing is a subset of the set of ear-like areas covered by the three-dimensional model of the outer ear.

3. The method according to claim 1, characterized in that The method of performing region segmentation on the three-dimensional outer ear model by using a three-dimensional segmentation algorithm based on deep learning to obtain the ear sample region to which each point in the three-dimensional outer ear model belongs includes: Using a point feature extraction network based on deep learning, the spatial position information of each point in the three-dimensional model of the outer ear is converted into a high-dimensional feature of each point; Using a local region feature extraction network based on deep learning, the geometric features of the local region are extracted from the three-dimensional model of the outer ear to obtain the features of the local region to which each point belongs; The feature fusion matrix is ​​used to fuse and classify the high-dimensional features of each point and the features of the local area to which it belongs, and the ear sample area to which each point belongs is obtained.

4. The method according to claim 1, wherein The method uses the high-dimensional features of the hearing aid housing three-dimensional model and the outer ear three-dimensional model as state variables, including: Extracting high-dimensional features of each point in the three-dimensional model of the hearing aid shell and the three-dimensional model of the outer ear from a three-dimensional segmentation algorithm based on deep learning; The high-dimensional features of each point in the two-dimensional model are pooled to obtain the high-dimensional features of the two-dimensional model.

5. The method according to claim 1, wherein The spatial transformation from any one of the two three-dimensional models to the other three-dimensional model as the action variable includes: According to the set unit translation distance and unit rotation angle, the translation transformation and rotation transformation along a single direction are divided into multiple discrete actions respectively; According to each discrete action, an action function for spatial transformation from any one of the two three-dimensional models to the other three-dimensional model is constructed.

6. The method according to claim 1, wherein The above-mentioned reward strategy uses the increasing similarity of the spatial positions of the two 3D models as a reward, determines the action variables of the next spatial transformation, and gradually realizes the spatial registration of the two 3D models, including: The Chamfer distance difference between the two 3D models before and after each spatial transformation is used as the reward function to determine the action variable for the next spatial transformation. Taking the maximum cumulative reward value as the optimization goal, the spatial alignment of two and three-dimensional models is gradually achieved.

7. The method according to claim 1, characterized in that Determining corresponding points of each point in the three-dimensional model of the hearing aid housing in the registered three-dimensional model of the outer ear includes: In the registered outer ear three-dimensional model, corresponding points closest to each point in the hearing aid housing three-dimensional model are determined.

8. An electronic device, characterized in that: include: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the region segmentation method for a three-dimensional hearing aid housing model according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the program is executed by a processor, the method for region segmentation of a three-dimensional model of a hearing aid housing according to any one of claims 1 to 7 is implemented.

Images