A method for judging the matching degree between the sound outlet direction of a hearing aid shell and the eardrum direction

Abstract

The embodiment of the present invention discloses a method for judging the degree of matching between the direction of the sound outlet of a hearing aid shell and the direction of the eardrum, and relates to the technical field of three-dimensional modeling of hearing aids. The method comprises: obtaining a three-dimensional model of a hearing aid shell to be judged, and a three-dimensional model of the outer ear from which it is derived; performing regional segmentation on the three-dimensional model of the hearing aid shell to identify the sound outlet cavity; performing spatial registration on the three-dimensional model of the hearing aid shell and the three-dimensional model of the outer ear to make the spatial positions of the two three-dimensional models most similar; determining the two curved cavities between the end face close to the eardrum and the sound outlet cavity in the three-dimensional model of the outer ear according to the positional relationship between the two three-dimensional models after registration; judging the degree of matching between the direction of the sound outlet cavity and the direction of the two curved cavities of the hearing aid shell and the direction of the eardrum. This embodiment can quickly, automatically and accurately judge whether the direction of the hearing aid shell is consistent with the direction of the sound outlet, thereby avoiding defects in the effect of the hearing aid shell.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of three-dimensional modeling of hearing aids, and in particular to a method for determining the degree of matching between the direction of a sound outlet hole of a hearing aid housing and the direction of an eardrum. Background Art

[0002] In the 3D modeling of hearing aids, a modeler usually reshapes the 3D ear sample model and finally completes the 3D model of the hearing aid shell. Whether the sound outlet of the shell is in the same direction as the user's eardrum will significantly affect the gain effect of the hearing aid when the user wears it. If the direction of the sound outlet and the direction of the eardrum are not consistent, not only will the hearing aid not be able to achieve the desired listening effect, but it will also cause misjudgment in the adjustment of the hearing aid parameters. Once a shell with an incorrect sound outlet direction enters the production and wearing process, there will be no detection mechanism to intercept the product, which not only wastes production resources but also has a significant negative impact on product quality.

[0003] In existing technology, the orientation of the sound hole is primarily manually adjusted by the modeler, who then visually verifies the orientation using 3D software. However, the original 3D ear sample and hearing aid housing are rendered in 2D within the 3D software. Verifying that the sound hole aligns with the eardrum requires observation from at least three angles, and this requires spatial visualization to determine consistency. This is extremely difficult for inexperienced modelers, and accuracy is uncertain.

[0004] Therefore, there is an urgent need for an effective method to determine the matching degree between the hearing aid shell and the direction of the sound outlet, and to automatically identify the hearing aid shell model after the modeler has modified it, so as to avoid physical defects in the hearing aid shell. Summary of the Invention

[0005] An embodiment of the present invention provides a method for determining the degree of matching between the direction of the sound outlet hole of a hearing aid housing and the direction of the eardrum, in order to solve the above technical problems.

[0006] In a first aspect, an embodiment of the present invention provides a method for determining the degree of matching between the direction of a sound outlet hole of a hearing aid housing and the direction of an eardrum, comprising:

[0007] Obtaining a three-dimensional model of the hearing aid shell to be judged and a three-dimensional model of the outer ear from which it is derived;

[0008] Performing regional segmentation on the three-dimensional model of the hearing aid housing to identify the sound hole cavity;

[0009] spatially registering the hearing aid housing three-dimensional model and the outer ear three-dimensional model so that the spatial positions of the two three-dimensional models are most similar;

[0010] Determining two curved cavities between the end surface close to the eardrum and the sound outlet cavity in the outer ear three-dimensional model according to the positional relationship between the two three-dimensional models after registration;

[0011] According to the directions of the sound outlet cavity and the two-curve cavity, the matching degree between the direction of the sound outlet of the hearing aid housing and the direction of the eardrum is determined.

[0012] In a second aspect, an embodiment of the present invention provides an electronic device, comprising:

[0013] one or more processors;

[0014] a memory for storing one or more programs,

[0015] When the one or more programs are executed by the one or more processors, the one or more processors implement the method for determining the degree of matching between the direction of the sound outlet hole of the hearing aid housing and the direction of the eardrum as described in any embodiment.

[0016] In a third aspect, an embodiment of the present invention further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the method for determining the degree of matching between the direction of the sound outlet hole of the hearing aid housing and the direction of the eardrum as described in any embodiment.

[0017] In summary, an embodiment of the present invention provides a method for judging the degree of matching between the direction of the sound hole of a hearing aid shell and the direction of the eardrum. First, a three-dimensional region segmentation algorithm based on deep learning is used to identify the sound hole area and the non-sound hole area in the three-dimensional model of the hearing aid shell; then, the three-dimensional model of the hearing aid shell and the three-dimensional model of the outer ear are spatially aligned to match their spatial positions; then, based on the spatial positions of the two models after alignment, the two-curved cavity in the three-dimensional model of the outer ear is identified as the basis for judging the axial direction of the user's eardrum; finally, based on the axial angle between the sound hole cavity and the two-curved cavity, whether the direction of the sound hole is qualified is accurately detected. The entire method is based on the three-dimensional model after the model is modified by the modeler, and can quickly realize the automatic detection of the direction of the sound hole with high accuracy, and the speed can be controlled within 150ms, which provides an important reference for the modeler to judge and adjust the direction of the sound hole; at the same time, the advance of the detection link greatly reduces unnecessary waste of resources in the subsequent process, and fully guarantees the production quality of the hearing aid shell. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 This is a flow chart of a method for determining the degree of matching between the direction of the sound outlet hole of a hearing aid housing and the direction of the eardrum, provided by an embodiment of the present invention;

[0020] Figure 2 is a schematic diagram of a three-dimensional model of the outer ear provided by an embodiment of the present invention;

[0021] Figure 3 is a schematic diagram of a three-dimensional model of a hearing aid housing provided by an embodiment of the present invention;

[0022] Figure 4 1 is a schematic diagram of a segmented sound outlet cavity provided by an embodiment of the present invention;

[0023] Figure 5 1 is a schematic diagram of the structure of an end-to-end deep learning network provided by an embodiment of the present invention;

[0024] Figure 6 1 is a schematic diagram of spatial registration of a three-dimensional model of an outer ear and a three-dimensional model of a hearing aid housing provided by an embodiment of the present invention;

[0025] Figure 7 Schematic diagram of a two-bend cavity provided by an embodiment of the present invention;

[0026] Figure 8 This is a schematic diagram of an obtuse triangle formed by the center point of the hearing aid housing 3D model, the center point of the sound outlet cavity, and the points in the outer ear 3D model, provided by an embodiment of the present invention.

[0027] Figure 9 is a schematic diagram of two spatial location clusters provided by an embodiment of the present invention;

[0028] Figure 10 Schematic diagram of determining a cavity end face using two spatial position clusters provided by an embodiment of the present invention;

[0029] Figure 11 This is a schematic diagram of a method of determining a cavity side surface using a preliminary axial direction determined by two end faces, provided by an embodiment of the present invention;

[0030] Figure 12 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0031] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention are described clearly and completely below. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are also within the scope of protection of the present invention.

[0032] In the description of the present invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limitations on the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0033] In the description of the present invention, it should also be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood broadly. For example, they may refer to fixed, detachable, or integral connections; mechanical or electrical connections; direct or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0034] Figure 1 This is a flow chart of a method for determining the degree of matching between the sound outlet direction of a hearing aid housing and the eardrum direction provided by an embodiment of the present invention. The method is executed by an electronic device, such as Figure 1 As shown, specifically including:

[0035] S110: Obtain a three-dimensional model of the hearing aid housing to be judged and a three-dimensional model of the outer ear from which the model is derived.

[0036] Among them, the outer ear three-dimensional model is the most original ear sample model obtained first in the production of hearing aids, such as Figure 2 Alternatively, a quick-curing material can be injected into the user's external auditory canal and auricle using a syringe. After solidification, the ear impression is taken out and then scanned in three dimensions using a three-dimensional scanner to obtain a digital three-dimensional model of the outer ear, which is also called an original three-dimensional ear sample.

[0037] Afterwards, the modeler will use 3D software to perform a series of customized processing on different parts of the original 3D ear sample, such as polishing, thickening, cutting, etc., and finally obtain the 3D model of the hearing aid shell (including the 3D model of the ear canal hearing aid shell and the 3D model of the behind-the-ear hearing aid ear mold), such as Figure 3 shown.

[0038] In this embodiment, for a certain user, a three-dimensional model of the hearing aid housing of the user is obtained as an object for determining the direction of the sound outlet, and a three-dimensional model of the outer ear of the user is simultaneously obtained as a basis for determining the direction of the user's eardrum.

[0039] S120: Segment the hearing aid housing three-dimensional model to identify the sound hole cavity.

[0040] This step performs regional segmentation on the hearing aid housing 3D model, dividing the entire 3D model into two areas: the sound hole area and the non-sound hole area. All points in the sound hole area together constitute the sound hole cavity. For example, the segmentation result is as follows: Figure 4 As shown in the figure, the purple cavity is the sound hole cavity.

[0041] The region segmentation process can be implemented manually based on experience or automatically using an end-to-end deep learning network. Figure 5 This is a structural diagram of this deep learning network. The input of this deep learning network is the point cloud data information of the three-dimensional model of the hearing aid shell, including the coordinates and normal vector of each point in the point cloud; the output is the judgment result of whether each point in the point cloud belongs to the sound hole area, with a value of 1 representing that it belongs to the sound hole area and 0 representing that it does not belong to the sound hole area.

[0042] For example, assume that the three-dimensional model of the hearing aid housing includes m points, and the coordinates of the i-th (i=1,2,…,m) point are , the normal vector coordinates of the point are , then the point cloud input of the deep learning network can be expressed as:

[0043]

[0044] Enter the above data into Figure 5 The deep learning network shown in Figure 1 performs region segmentation. The point feature extraction network converts the input 3D point cloud data into a representation in a high-dimensional feature space through layer-by-layer convolution operations. Specifically, the formula for the convolution operation is:

[0045]

[0046] in, Represents the spatial position information of the i-th point in the input point cloud, including the coordinates and normal vector of the point; Represents the data corresponding to the point after the convolution operation, represents the weight vector, represents the kernel function of the convolution kernel, Represents the kernel function centered at the i-th point The spatial position information of the covered points, N represents the kernel function centered on the i-th point The number of points covered. After a series of convolutional layers, the data of each point are mapped into a high-dimensional feature vector This feature is an implicit feature that contains the spatial information of the point, providing accurate information on the point granularity for subsequent end-to-end point-by-point region segmentation. Optionally, a multi-step 1×1 convolution kernel can be used to extract the above point features. Specify a convolution kernel size of 1×1, an input dimension of 6, and an output dimension equal to the dimension of the high-dimensional feature.

[0047] The local area feature extraction network consists of multiple layers of sparse convolutional layers, pooling layers, and fully connected layers. Sparse convolution is the main component of the local area feature extraction network. After the input point cloud enters the local area feature extraction network, each convolution layer extracts the local geometric features of the shell surface through the sliding operation of the sparse convolution kernel (for example, a 6×6×6 convolution kernel). Specifically, the formula for each sparse convolution is as follows:

[0048]

[0049] in, represents the number of non-empty locations in the input point cloud, and Represent the weight and kernel function of sparse convolution respectively, Represents the kernel function centered at the i-th point in the input point cloud The spatial location information except for the non-empty locations covered, Represents the data corresponding to the point after the sparse convolution operation.

[0050] After sparse convolution, the pooling layer downsamples the features extracted by the sparse convolution layer, reducing the amount of data while retaining key features. Optionally, the number of pooling layers can be fewer than the number of sparse convolution layers. Generally speaking, the more complex the ear mold shape, the fewer pooling layers can be used. The fully connected layer integrates the features extracted by the pooling layer to obtain a vector that represents the characteristics of the local region of the 3D ear sample.

[0051] The local region feature extraction network ultimately outputs a feature vector for the local region of each point in the point cloud data, such as a 512-dimensional feature vector. This vector reflects the local geometric characteristics of the outer ear. It should be noted that the local region here refers to the spatial range surrounding a point, and is not the two types of regions to be identified, so this distinction should be made.

[0052] After the two feature extraction networks complete feature extraction, they are combined through an additional feature fusion matrix , perform weighted fusion of the feature vector of each point in the point cloud data and the features of the local area where it is located to obtain the final feature data of each point. A point can be understood as a matrix of the number of point clouds × 2, which is a learnable parameter matrix. The dimension 2 corresponds to the weights of point features and local features respectively. During the training process, the model learns whether each point in the point cloud relies more on point information or local information.

[0053] Finally, the final feature vector of each point is input into the multi-layer perceptron to classify each point into the sound hole area and the non-sound hole area. The final output of the model is ,in, , which represents the judgment result of whether the i-th point in the point cloud of the hearing aid shell three-dimensional model belongs to the sound outlet area.

[0054] S130 , spatially aligning the hearing aid housing three-dimensional model and the outer ear three-dimensional model so that the spatial positions of the two three-dimensional models are most similar.

[0055] This step performs spatial registration on the hearing aid shell 3D model and the outer ear 3D model from which it is derived, to maximize the spatial similarity between the two models. Figure 6 The image shows a registration result, with the registered 3D outer ear model marked in red and the registered 3D hearing aid housing model marked in black. Spatial registration is achieved when the spatial position and orientation of the same ear region in the two models are most similar (for example, the first and second bends are in the same location and have similar shapes).

[0056] To achieve the above-mentioned registration effect, this embodiment provides a reinforcement learning method, which uses the characteristics of the three-dimensional model of the hearing aid shell and the three-dimensional model of the outer ear as state variables, and uses the spatial transformation from any one of the two three-dimensional models to the other as an action variable to construct a reinforcement learning model. Through reinforcement learning, the three-dimensional model is gradually spatially transformed, and after each step of the spatial transformation, the action variable for the next spatial transformation is determined based on the increasing spatial morphological similarity of the two three-dimensional models as a reward strategy, thereby gradually achieving spatial registration of the two three-dimensional models.

[0057] In a specific embodiment, the point cloud PO (including n data points) of the outer ear three-dimensional model can be expressed as:

[0058]

[0059] The point cloud PW (including m data points) of the hearing aid shell 3D model can be expressed as:

[0060]

[0061] Based on the above point cloud data, the state variables and action variables of the reinforcement learning model can be constructed.

[0062] Specifically, for state variables, feature embedding can be used to convert the two sets of point cloud data into global feature vectors, and the two global feature vectors can be used to generate state variables. Optionally, feature calculation can be performed on the outer ear 3D model PO to construct a high-dimensional feature fo; feature calculation can be performed on the hearing aid shell 3D model PW to construct a high-dimensional feature fw; then the state variable Status can be expressed as:

[0063] Statu=concat(fo,fw)

[0064] Here, concat(,) represents a concatenation operation. Optionally, feature calculation can still use the same point feature extraction network structure as in S120, with a max pooling layer added after it. By taking the maximum value of all points in each feature dimension, high-dimensional features for the entire point cloud data can be obtained. For example, the dimensions of fo and fw can be 1024.

[0065] At the same time, for action variables, the spatial transformation from any one of the two three-dimensional models to the other three-dimensional model can be used as the action space, and the action space can be divided to obtain various action variables.

[0066] Optionally, first perform direction division to divide the three-dimensional space into a set of three directions Direct-Space=[X,Y,Z].

[0067] Then, based on the set unit translation distance, the translation action space along a single direction is divided into multiple discrete translation actions. For example, assuming that the minimum translation distance in the three-dimensional space is 0.05, multiple discrete translation distances can be divided based on the minimum translation distance. The translation action space Lim_T can be expressed as:

[0068] Lim_T=[-0.2, -0.1, -0.05, 0, 0.05, 0.1, 0.2].

[0069] Similarly, according to the set unit rotation angle, the rotation action space along a single direction is divided into multiple discrete rotation angles, and the rotation action space Lim_R can be expressed as:

[0070] Lim_R=Lim_T×π / 2=[-π / 10, -π / 20, -π / 40, 0, π / 40, π / 20, π / 10].

[0071] The above direction division, as well as the translation and rotation division can be simply expressed as the direction space [0, 1, 2] and the scale space [0, 1, 2, 3, 4, 5, 6, 7]. Each combination of direction, translation scale and rotation scale constitutes an action variable.

[0072] Furthermore, based on each action variable, an action function for spatial transformation from one of the two three-dimensional models to the other three-dimensional model can be constructed. The action function can be expressed as a combination of a translation function and a rotation function.

[0073] Among them, the translation function Translate(Direct, Lim) represents translation along a certain direction. The direction of translation is the direction represented by the Direct element in Direct-Space, and the translation distance is The following explanation takes the spatial transformation of PO as an example, and the translated point cloud PO1'=PO+Translate(Direct, Lim).

[0074] The rotation function Rotate(Direct, LIM_R) represents rotation along a certain direction. The direction of rotation is the direction represented by the Direct element in Direct-Space, and the angle of rotation is The angle corresponding to the Limth element in . The rotated point cloud PO2'=Rotate(Direct,Lim)PO.

[0075] Furthermore, the two functions can be expressed in matrix form as follows:

[0076]

[0077]

[0078] Now that the state variables and action variables have been constructed, we can proceed to construct the reward function of the reinforcement learning model. Optionally, this embodiment uses the Chamfer distance between two point cloud sets to characterize the spatial similarity between two 3D models. The calculation formula for the Chamfer distance is:

[0079]

[0080] Among them, U and V are the two point cloud sets to be compared, u and v represent the points in U and V respectively. Represents the number of points in the point cloud set U. The smaller the Chamfer distance between two point cloud sets, the higher the similarity. Based on the Chamfer distance, the following reward function can be constructed:

[0081] Reward=CD(PO,PW)-CD(PO',PW)

[0082] Where Reward represents the reward value, PO' represents the PO after spatial transformation, CD(PO, PW) represents the Chamfer distance between PO and PW before spatial transformation, and CD(PO', PW) represents the Chamfer distance between PO and PW after spatial transformation. Under this reward function, if the spatial transformation shortens the Chamfer distance between the two 3D models, the current action is rewarded positively; if the spatial transformation shortens the Chamfer distance between the two 3D models, the current action is rewarded negatively, thereby encouraging the two 3D models to move closer together in the direction of decreasing Chamfer distance.

[0083] Based on the above state variables, action variables, and reward functions, reinforcement learning training can be performed. The reinforcement learning model includes an actor network and a decision network. The feature extraction network that generates the above state variable Status can be used as the actor network, and Status is the output of the actor network. The decision network can use a multi-layer perceptron structure or a linear-based transformer network structure to evaluate each action variable based on Status and reward functions and determine the next action variable. During training, the reward value is accumulated each time:

[0084] Rewards=SUM(CD(PO,PW)-CD(Dot(Rotate[Lim],PO)+Translate[Lim],PW))

[0085] Where Dot represents the dot product of the matrix. When the cumulative reward value Rewards no longer increases, it can be determined that the two models have been aligned, that is, Figure 6 The point cloud PWO after PO registration can be expressed as:

[0086] PWO= PO+

[0087] in, and They represent the rotation matrix and translation matrix of the e-th step spatial transformation respectively.

[0088] It should be noted that the above S120 and S130 are independent of each other, and can be performed in a reversed order or simultaneously, which is not specifically limited in this embodiment.

[0089] S140. Determine two curved cavities between the end surface close to the eardrum and the sound outlet cavity in the three-dimensional model of the outer ear according to the positional relationship between the two registered three-dimensional models.

[0090] The two-bend cavity to be identified in this step is as follows Figure 7 As shown in the red box. For details, refer to Figure 7 After the two 3D models are registered, the area between the end surface of the outer ear model closest to the eardrum and the sound outlet cavity of the hearing aid housing is the two-curve cavity. This cavity extends along the two-curve direction of the ear canal and is very close to the direction of the user's eardrum, which can be used as a basis for subsequent determination of the user's eardrum orientation.

[0091] In a specific embodiment, each point U in the three-dimensional model of the external ear after registration can be i The following operations are performed respectively: the center point wc1 of the hearing aid housing 3D model after registration, the center point wc2 of the sound outlet cavity after registration, and U i , forming a triangle, such as Figure 8 If the angle of the triangle with the center point wc2 of the sound hole cavity as the vertex is an obtuse angle, it can be determined that the current U i It belongs to the two-bend cavity.

[0092] Furthermore, the distance from wc1 to wc2 is recorded as L1, U i The distance to wc1 is L2, U i The distance to wc2 is L3; if the triangle satisfies "the triangle is an obtuse triangle, and the side corresponding to L2 is the longest side of the triangle", then U i is a point on the two-bend cavity.

[0093] For all U i After making the above judgments, the point cloud of the two-bend cavity can be obtained.

[0094] S150: Determine the degree of matching between the direction of the sound outlet hole of the hearing aid housing and the direction of the eardrum according to the directions of the sound outlet hole cavity and the two-curve cavity.

[0095] Through the operations of S110-S140, this embodiment obtains two cavities: one is the sound outlet cavity from the three-dimensional model of the hearing aid housing, and the other is the two-curve cavity from the three-dimensional model of the outer ear. The axial direction of the sound outlet cavity represents the direction of the sound outlet of the hearing aid housing, and the axial direction of the two-curve cavity area represents the direction of the user's eardrum. This step will accurately estimate the axial directions of the two cavities and determine the matching degree between the sound outlet of the hearing aid housing and the eardrum based on the axial direction.

[0096] In a specific embodiment, two cavities can be processed separately by the same method to obtain the axial directions of the two cavities. Taking any one of the cavities as an example, the method includes the following steps:

[0097] Step 1: Use voxel downsampling to downsample the original point cloud of the cavity to be processed to reduce the amount of subsequent calculations. Optionally, a voxel size p is pre-set, and the three-dimensional space where the point cloud resides is divided into a number of voxels based on this voxel size. Each point in the point cloud data is then assigned to a corresponding voxel. For each voxel, the average coordinate value of all points within the voxel is taken as the representative point of the voxel, and the downsampled point cloud is composed of the representative points of all voxels.

[0098] Step 2: Determine the two end faces of the cavity based on the large curvature points of the downsampled point cloud. i , calculate the curvature of the point respectively. The specific method is: take point P i Determine a neighborhood with a radius of R as the center, and fit a plane based on the down-sampling points in the neighborhood. The plane equation is , where x, y, z are three-dimensional space coordinates, and a, b, c, d are fitted plane parameters; the normal vector of the plane can be expressed as (a, b, c), P i The curvature k can be calculated according to the following formula:

[0099]

[0100] Furthermore, the plane fitting process can be realized by the least squares method. In the least squares method, according to the neighborhood point set N i Each point in , construct the objective function By taking partial derivatives of a, b, c, d and setting them to 0, we can get the fitting parameters a, b, c, d of the plane.

[0101] After the curvature calculation is complete, the high curvature points in the cavity are screened based on the set curvature threshold. These high curvature points are the points where the shape of the cavity is most significant, and are concentrated at the intersection of the cavity side and the cavity ends. Of course, for irregular cavities, the cavity side will also include some high curvature points.

[0102] After the screening is completed, all large curvature points are divided into two categories through spatial clustering. Optionally, the Kmeans clustering method can be used to cluster the spatial locations of large curvature points, so that large curvature points with similar spatial locations are clustered into one category, and finally two spatial location clusters are obtained. Figure 9The blue cylinder simply illustrates a cavity, while the red and green boxes respectively indicate the ranges of the two spatial location clusters. Furthermore, in the Kmeans clustering method, two initial centroids can be randomly selected from the points of large curvature. Each point is then assigned to the nearest centroid, and the centroid of each cluster (i.e., the average position of all points in the cluster) is recalculated. Each point is then reassigned to the nearest new centroid, and so on. This cycle repeats until the final centroids no longer change, resulting in the final clusters and cluster centers.

[0103] After clustering is completed, a set proportion of points that are farthest from the center point of the entire point cloud can be selected from each of the two clusters to form two new point cloud groups. For example, the point cloud composed of all large curvature points is recorded as B. For each cluster, the distance between each point in the cluster and the center point of B is calculated, and the distances are arranged from large to small. The points ranked in the top 1 / 3 are taken as a new point cloud group, thereby obtaining two new point cloud groups, such as Figure 10 Then, the center points of the two new point cloud groups are used as two new anchor points. and ; Calculation process and straight line , and determine the passing point ,by Is the plane F1 with normal direction, and the point ,by If F2 is the plane in the normal direction, then F1 and F2 are the two end faces of the cavity.

[0104] Of course, the voxel downsampling in step 1 can be omitted, and the process can be directly turned to step 2 to determine the two end faces of the cavity based on the large curvature points in the cavity.

[0105] Step 3: Determine the preliminary axis of the two-curve cavity based on the center points of the two end faces; and determine the points on the side of the two-curve cavity based on the angle between the normal vector of each point in the two-curve cavity and the preliminary axis. Optionally, after obtaining the two end faces, the preliminary axis of the cavity can be determined based on the center points of the two end faces. Taking the above end faces F1 and F2 as an example, and can be regarded as their respective center points, It is the preliminary axis of the cavity. Points in the cavity that are located between the two end faces and whose normal vectors have an angle with the preliminary axis greater than a set threshold can be screened out. These points are the points on the side of the cavity. For example, refer to Figure 11 , the blue arrows represent the normal vectors of each point in the cavity, then for each point T in the cavity point cloud i, if T i The normal vector and If the angle between T and i is a point on the side of the cavity. The angle range is [0, 90°]. For angles greater than 90°, the result can be subtracted from 180°.

[0106] Step 4: Move the points on the side of the cavity gradually in the direction opposite to the normal vector of each point until the point density in the neighborhood of each point decreases after the movement. Optionally, take the minimum moving distance in three-dimensional space as the step length and move each point on the cavity. , respectively along Move in the opposite direction of the normal vector (i.e., shrink the points of the entire cavity radially inward to make them closer to the central axis of the cavity), and for each point , determine a The neighborhood of the center, The neighborhood moves before and after the movement, but the neighborhood radius r remains unchanged. After each move has completed one step, a round of movement is completed, and the following operations are performed:

[0107] Compare each point The number of cavity points in the neighborhood before and after the movement is K1 and K2; if the number of cavity points in the neighborhood after the movement is less than the number K1 before the movement, it means that the point has moved excessively and crossed the cavity axis; in this case, Go back one step and do not change in subsequent loops After executing the above judgment for each point, the next round of movement begins, and the cycle repeats until the positions of all points no longer change. Finally, a new point set is formed. , this point set is very close to the axis of the cavity.

[0108] Step 5: Based on the new point set , fitting the axial straight line of the two-curve cavity. Optionally, the least square method can be used to fit the point set For straight line estimation, the point-wise equation of the straight line can be expressed as:

[0109]

[0110] Among them, x, y and z are variables, 、 and t are the parameters to be fitted, 、 and are the coordinates of known data points.

[0111] In order to fit the above parameters, the following intermediate variables k1, b1, k2 and b2 can be constructed:

[0112]

[0113] Then the above space straight line can be expressed as:

[0114]

[0115] Point set The coordinates of each point in ( , , ) as 、 and , substituting into the above equation, we can get the sum of squares of the residuals of these points and the two planes and :

[0116]

[0117] Among them, Q represents the point cloud The number of midpoints. According to the least squares method, we need to achieve and Therefore, according to the above equation, we can differentiate k1, b1, k2 and b2 respectively and set the derivative to 0 to solve k1, b1, k2 and b2, and then calculate the axial straight line equation parameters of the two-bend cavity. 、 and t, the direction vector of the axial line in three-dimensional space is .

[0118] After performing the above steps 1 to 5 on the two-bend cavity and the sound hole cavity respectively, the axial straight line of the two-bend cavity can be obtained respectively. Compared with the preliminary axial line obtained above, the straight line is more accurate and can better reflect the direction of the cavity.

[0119] Finally, the matching degree between the direction of the sound outlet of the hearing aid shell and the direction of the eardrum is determined based on the axial straight line of the two-curve cavity and the axial straight line of the sound outlet cavity. :

[0120]

[0121] in, , represents the axial vector of the two-bend cavity; , represents the axial vector of the sound hole cavity. If the angle If the angle is less than a certain threshold, the direction of the sound hole is determined to be normal; if the angle is greater than or equal to the threshold, the direction of the sound hole is determined to be abnormal.

[0122] In summary, this embodiment provides a method for judging the degree of matching between the direction of the sound outlet of a hearing aid shell and the direction of the eardrum. First, a three-dimensional region segmentation algorithm based on deep learning is used to identify the sound outlet area and the non-sound outlet area in the three-dimensional model of the hearing aid shell; then, the three-dimensional model of the hearing aid shell and the three-dimensional model of the outer ear are spatially aligned to match their spatial positions; then, based on the spatial positions of the two models after alignment, the two-curved cavity in the three-dimensional model of the outer ear is identified as the basis for judging the axial direction of the user's eardrum; then, the sound outlet cavity and the two-curved cavity are respectively retracted by radially retracting the point cloud, and point clouds that are almost parallel to the axes of the two cavities are obtained respectively, and axial straight lines of the two cavities are fitted based on these point clouds to ensure the accuracy of the axial estimation; finally, based on the axial angle between the sound outlet cavity and the two-curved cavity, whether the sound outlet direction is qualified is accurately detected. The entire method is based on the three-dimensional model modified by the mold maker, and can quickly realize automatic detection of the direction of the sound hole with high accuracy and a speed that can be controlled within 150ms, providing an important reference for the mold maker to judge and adjust the direction of the sound hole; at the same time, the pre-detection link greatly reduces unnecessary resource waste in the subsequent process, fully ensuring the production quality of the hearing aid shell.

[0123] Figure 12 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention is shown in FIG. Figure 12 As shown, the device includes a processor 60, a memory 61, an input device 62 and an output device 63; the number of processors 60 in the device can be one or more. Figure 12 In the embodiment, a processor 60 is used as an example; the processor 60, the memory 61, the input device 62 and the output device 63 in the device can be connected by a bus or other means. Figure 12 The bus connection is taken as an example.

[0124] Memory 61, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the method for determining the degree of match between the direction of the sound outlet opening of a hearing aid housing and the direction of the eardrum in the embodiments of the present invention. Processor 60 executes the software programs, instructions, and modules stored in memory 61 to execute various functional applications and data processing of the device, thereby implementing the aforementioned method for determining the degree of match between the direction of the sound outlet opening of a hearing aid housing and the direction of the eardrum.

[0125] The memory 61 may primarily include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function; the data storage area may store data generated based on the use of the terminal. Furthermore, the memory 61 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state memory device. In some instances, the memory 61 may further include memory remotely located relative to the processor 60, and these remote memories may be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0126] The input device 62 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the device. The output device 63 may include a display device such as a display screen.

[0127] An embodiment of the present invention further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the method for determining the degree of matching between the direction of the sound outlet hole of a hearing aid housing and the direction of the eardrum according to any embodiment.

[0128] The computer storage medium of the embodiments of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or device.

[0129] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0130] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0131] Computer program code for performing the operations of the present invention can be written in one or more programming languages, or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as C or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0132] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the technical solutions of the embodiments of the present invention.

Claims

1. A method for determining the degree of matching between the direction of the sound outlet of a hearing aid housing and the direction of the eardrum, characterized in that: include: Obtaining a three-dimensional model of the hearing aid shell to be judged and a three-dimensional model of the outer ear from which it is derived; Performing regional segmentation on the three-dimensional model of the hearing aid housing to identify the sound hole cavity; spatially registering the hearing aid housing three-dimensional model and the outer ear three-dimensional model so that the spatial positions of the two three-dimensional models are most similar; Constructing triangles using the center of the registered three-dimensional model of the hearing aid housing, the center of the sound outlet cavity, and each point in the registered three-dimensional model of the outer ear; if the angle of each triangle with the center point of the sound outlet cavity as the vertex is an obtuse angle, the points in the registered three-dimensional model of the outer ear are determined to belong to the two-curve cavity; Each point on the side of the two-curve cavity is gradually moved in the direction opposite to the normal vector of each point until the point density in the neighborhood of each point decreases after the movement; the axial straight line of the two-curve cavity is fitted according to the position of each point before the last step of movement; based on the axial straight line of the two-curve cavity and the axial straight line of the sound outlet cavity, the matching degree between the direction of the sound outlet of the hearing aid shell and the direction of the eardrum is judged.

2. The method according to claim 1, characterized in that Before gradually moving each point on the side surface of the two-bend cavity in a direction opposite to the normal vector of each point, the method further includes: Determining two end faces of the two-bend cavity according to a point of large curvature in the two-bend cavity; Determine the preliminary axial direction of the two-bend cavity according to the center points of the two end faces; The points on the side surface of the two-bend cavity are determined according to the angle between the normal vector of each point in the two-bend cavity and the preliminary axis.

3. The method according to claim 2, characterized in that Determining two end faces of the two-bend cavity according to the large curvature point in the two-bend cavity includes: Screening large curvature points in the two-curve cavity according to a set curvature threshold; Through spatial clustering, the major curvature points are divided into two categories; The planes formed by the two clusters are respectively used as the two end faces of the two-bend cavity.

4. The method according to claim 3, characterized in that The method of screening the large curvature points in the two-bend cavity according to the set curvature threshold comprises: Dividing the two-curve cavity into voxels, and forming a new point cloud of the two-curve cavity with each voxel; According to the set curvature threshold, large curvature points are screened from the new point cloud.

5. The method according to claim 1, wherein The performing of region segmentation on the three-dimensional model of the hearing aid housing to identify the sound hole cavity includes: Using a point feature extraction network based on deep learning, the spatial position information of each point in the three-dimensional model of the hearing aid housing is converted into a high-dimensional feature of each point; Using a local area feature extraction network based on deep learning, the geometric features of the local area are extracted from the three-dimensional model of the hearing aid housing to obtain the features of the local area to which each point belongs; The feature fusion matrix is used to fuse and classify the high-dimensional features of each point and the features of the local area to which it belongs, and determine whether each point belongs to the sound hole cavity.

6. The method according to claim 1, characterized in that The spatial registration of the hearing aid housing three-dimensional model and the outer ear three-dimensional model so that the spatial positions of the two three-dimensional models are most similar includes: Constructing a reinforcement learning model using the high-dimensional features of the hearing aid housing model and the outer ear model as state variables and the spatial transformation from one of the two three-dimensional models to the other as an action variable; Based on the reinforcement learning model, any three-dimensional model is gradually spatially transformed, and after each step of spatial transformation, the action variable of the next spatial transformation is determined by taking the increase in the spatial position similarity of the two three-dimensional models as a reward strategy, thereby gradually achieving spatial alignment of the two three-dimensional models.

7. An electronic device, characterized in that: include: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method for determining the degree of matching between the direction of the sound outlet hole of the hearing aid housing and the direction of the eardrum according to any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the program is executed by a processor, the method for determining the degree of matching between the direction of the sound outlet hole of the hearing aid housing and the direction of the eardrum as claimed in any one of claims 1 to 6 is implemented.

Images