Method for judging matching degree of sound outlet hole direction and tympanic membrane direction of hearing aid shell

Abstract

The embodiment of the invention discloses a method for judging the matching degree of the direction of a sound outlet hole of a hearing aid shell and the direction of a tympanic membrane, and relates to the technical field of hearing aid three-dimensional modeling. The method comprises the following steps: acquiring a hearing aid shell three-dimensional model to be judged and an external ear three-dimensional model of a source of the hearing aid shell three-dimensional model; performing region segmentation on the hearing aid shell three-dimensional model, and identifying a sound hole cavity; performing spatial registration on the hearing aid shell three-dimensional model and the external ear three-dimensional model to enable the spatial positions of the two three-dimensional models to be most similar; according to the position relation of the two registered three-dimensional models, two bent cavities between the end face, close to the tympanic membrane, in the external ear three-dimensional model and the sound outlet hole cavity are determined; and judging the matching degree between the direction of the sound outlet hole of the hearing aid shell and the direction of the tympanic membrane according to the directions of the sound outlet hole cavity and the two-way bent cavity. The embodiment of the invention can quickly, automatically and accurately judge whether the hearing aid shell is consistent with the direction of the sound outlet hole, thereby avoiding the effect defect of the hearing aid shell.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of three-dimensional modeling of hearing aids, and in particular to a method for judging the matching degree between the sound outlet direction of a hearing aid shell and the eardrum direction. Background Art

[0002] In the three-dimensional modeling of hearing aids, usually a mold modifier modifies the three-dimensional ear model, and finally completes the three-dimensional model of the hearing aid shell. Whether the sound outlet of this shell faces the same direction as the user's eardrum will significantly affect the gain effect of the hearing aid when the user wears it. If the sound outlet direction and the eardrum position direction are not unified, not only the due listening effect of the hearing aid cannot be exerted, but also misjudgment will be caused for the parameter adjustment of the hearing aid. Once the shell with the wrong sound outlet direction enters the production and wearing links, there will be no detection mechanism to intercept this product, which not only wastes production resources, but also has an obvious negative impact on the product quality.

[0003] In the prior art, the sound outlet direction is mainly manually adjusted by the mold modifier, and after the modification, an experienced mold modifier uses three-dimensional software for visual judgment. The original three-dimensional ear model and the hearing aid shell are presented in a two-dimensional manner in the three-dimensional software, and it is necessary to observe from at least three angles to confirm whether the sound outlet is in a straight line with the eardrum direction. During this period, spatial imagination is also required to judge the consistency, which is very difficult for mold modifiers with insufficient experience, and the accuracy of the judgment cannot be guaranteed.

[0004] Therefore, there is an urgent need for an effective method for judging the matching degree between the hearing aid shell and the sound outlet direction to automatically discriminate the hearing aid shell model after the mold modifier has modified it, so as to avoid the effect defects of the hearing aid shell at the physical level. Summary of the Invention

[0005] The embodiments of the present invention provide a method for judging the matching degree between the sound outlet direction of a hearing aid shell and the eardrum direction to solve the above technical problems.

[0006] In a first aspect, the embodiments of the present invention provide a method for judging the matching degree between the sound outlet direction of a hearing aid shell and the eardrum direction, including:

[0007] Obtain the three-dimensional model of the hearing aid shell to be judged and the three-dimensional model of the outer ear from which it is derived;

[0008] Perform regional segmentation on the three-dimensional model of the hearing aid shell to identify the sound outlet cavity;

[0009] Perform spatial registration on the three-dimensional model of the hearing aid shell and the three-dimensional model of the outer ear to make the spatial positions of the two three-dimensional models most similar;

[0010] Determine the second-bend cavity between the end face of the outer ear three-dimensional model close to the eardrum and the sound outlet hole cavity according to the positional relationship between the two three-dimensional models after registration;

[0011] Judge the matching degree between the direction of the sound outlet hole of the hearing aid shell and the direction of the eardrum according to the trends of the sound outlet hole cavity and the second-bend cavity.

[0012] In a second aspect, an embodiment of the present invention provides an electronic device, which includes:

[0013] One or more processors;

[0014] A memory for storing one or more programs,

[0015] When the one or more programs are executed by the one or more processors, the one or more processors implement the method for judging the matching degree between the direction of the sound outlet hole of the hearing aid shell and the direction of the eardrum according to any embodiment.

[0016] In a third aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the method for judging the matching degree between the direction of the sound outlet hole of the hearing aid shell and the direction of the eardrum according to any embodiment.

[0017] In summary, the embodiment of the present invention provides a method for judging the matching degree between the direction of the sound outlet hole of the hearing aid shell and the direction of the eardrum. First, use a three-dimensional region segmentation algorithm based on deep learning to identify the sound outlet hole region and the non-sound outlet hole region in the three-dimensional model of the hearing aid shell; then perform spatial registration on the three-dimensional model of the hearing aid shell and the three-dimensional model of the outer ear to make their spatial positions match; then identify the second-bend cavity in the three-dimensional model of the outer ear according to the spatial positions of the two models after registration as the basis for judging the axial direction of the user's eardrum; finally, accurately detect whether the direction of the sound outlet hole is qualified according to the axial included angle between the sound outlet hole cavity and the second-bend cavity. The entire method is based on the three-dimensional model after the mold repairer repairs the mold, and can quickly realize the automatic detection of the sound outlet hole direction, with high accuracy and the speed can be controlled within 150 ms, providing an important reference for the mold repairer to judge and adjust the sound outlet hole direction; at the same time, the preposition of the detection link greatly reduces the unnecessary resource waste in the subsequent process and fully guarantees the production quality of the hearing aid shell. Description of the Drawings

[0018] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained according to these drawings.

[0019] Figure 1 It is a flowchart of a method for judging the matching degree between the sound outlet hole direction of a hearing aid shell and the eardrum direction provided by an embodiment of the present invention;

[0020] Figure 2 It is a schematic diagram of a three-dimensional model of the outer ear provided by an embodiment of the present invention;

[0021] Figure 3 It is a schematic diagram of a three-dimensional model of a hearing aid shell provided by an embodiment of the present invention;

[0022] Figure 4 It is a schematic diagram of the segmentation of the sound outlet hole cavity provided by an embodiment of the present invention;

[0023] Figure 5 It is a schematic diagram of the structure of an end-to-end deep learning network provided by an embodiment of the present invention;

[0024] Figure 6 It is a schematic diagram of the spatial registration of a three-dimensional model of the outer ear and a three-dimensional model of a hearing aid shell provided by an embodiment of the present invention;

[0025] Figure 7 It is a schematic diagram of a two-bend cavity provided by an embodiment of the present invention;

[0026] Figure 8 It is a schematic diagram of an obtuse triangle formed by the center point of the three-dimensional model of the hearing aid shell, the center point of the sound outlet hole cavity, and a point in the three-dimensional model of the outer ear provided by an embodiment of the present invention

[0027] Figure 9 It is a schematic diagram of two spatial position clustering clusters provided by an embodiment of the present invention;

[0028] Figure 10 It is a schematic diagram of determining the end face of the cavity by using two spatial position clustering clusters provided by an embodiment of the present invention;

[0029] Figure 11 It is a schematic diagram of determining the side surface of the cavity by using the preliminary axis determined by two end faces provided by an embodiment of the present invention;

[0030] Figure 12 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Detailed implementation manners

[0031] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be described clearly and completely below. Apparently, the described embodiments are only a part rather than all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the scope protected by the present invention.

[0032] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as a limitation to the present invention. In addition, the terms "first", "second", "third" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance.

[0033] In the description of the present invention, it should also be noted that unless otherwise clearly defined and limited, the terms "installed", "connected", "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0034] Figure 1 is a flowchart of a method for judging the matching degree between the sound outlet hole direction of a hearing aid shell and the eardrum direction provided by an embodiment of the present invention. This method is executed by an electronic device, such as Figure 1 shown, and specifically includes:

[0035] S110. Obtain the three-dimensional model of the hearing aid shell to be judged and the three-dimensional model of the outer ear from which it is derived.

[0036] Among them, the three-dimensional model of the outer ear is the most original ear sample model obtained first in the manufacture of hearing aids, such as Figure 2 shown. Optionally, a rapid-curing material can be injected into the user's external auditory canal to the auricle part using a syringe. After it solidifies, the ear impression is taken out, and then the ear impression of the wearer is three-dimensionally scanned using a three-dimensional scanner to obtain a digital three-dimensional model of the outer ear, which is also called the original three-dimensional ear sample.

[0037] After that, the mold repairer will use 3D software to perform a series of customized processing operations such as grinding, thickening, and cutting on different parts of the original 3D ear sample, and finally obtain the 3D model of the hearing aid shell (including the 3D model of the in-the-ear hearing aid shell and the ear mold 3D model of the behind-the-ear hearing aid), as Figure 3 shown.

[0038] In this embodiment, for a certain user, the 3D model of the hearing aid shell of this user is obtained as the object for judging the sound outlet hole direction, and at the same time, the 3D model of the outer ear of this user is obtained as the basis for judging the eardrum direction of the user.

[0039] S120. Perform region segmentation on the 3D model of the hearing aid shell to identify the sound outlet hole cavity.

[0040] In this step, region segmentation is performed on the 3D model of the hearing aid shell, and the entire 3D model is segmented into two regions: the sound outlet hole region and the non-sound outlet hole region. All points in the sound outlet hole region together constitute the sound outlet hole cavity. Exemplarily, the segmentation result is as Figure 4 shown, and the purple cavity in the figure is the sound outlet hole cavity.

[0041] This region segmentation process can be achieved manually according to experience or automatically using an end-to-end deep learning network. Optionally, Figure 5 is a schematic structural diagram of such a deep learning network. The input of this deep learning network is the point cloud data information of the 3D model of the hearing aid shell, including the coordinates and normal vectors of each point in the point cloud; the output is the judgment result of whether each point in the point cloud belongs to the sound outlet hole region, with a value of 1 representing belonging to the sound outlet hole region and 0 representing not belonging to the sound outlet hole region.

[0042] Exemplarily, assume that the 3D model of the hearing aid shell includes m points, and the coordinates of the i-th (i = 1, 2,..., m) point are and the normal vector coordinates of this point are , then the point cloud input of the deep learning network can be expressed as:

[0043]

[0044] Input the above data into Figure 5 the deep learning network shown for region segmentation. Among them, the point feature extraction network converts the input 3D point cloud data into a representation in a high-dimensional feature space through layer-by-layer convolution operations. Specifically, the formula for the convolution operation is:

[0045]

[0046] Among them, represents the spatial position information of the i-th point in the input point cloud, including the coordinates and normal vector of this point; Represents the data corresponding to this point after the convolution operation, represents the weight vector, represents the kernel function of the convolution kernel, represents the kernel function centered on the i-th point The spatial position information of the points covered by, N represents the kernel function centered on the i-th point The number of points covered. After a series of convolutional layers, the data of each point are all mapped to a high-dimensional feature vector , this feature is a latent feature containing the spatial information of this point, providing accurate point-level information for subsequent end-to-end per-point region segmentation. Optionally, the above-mentioned point feature extraction can be implemented using multi-step 1×1 convolution kernels. Specify the convolution kernel size as 1×1 and the input dimension as 6, and the output dimension as the dimension of the high-dimensional feature.

[0047] The local region feature extraction network includes multiple layers of sparse convolutional layers, pooling layers, and fully connected layers. Among them, sparse convolution is the main body of the local region feature extraction network. After the input point cloud enters the local region feature extraction network, the local geometric features of the shell surface are extracted through the sliding operation of the sparse convolution kernel (such as a 6×6×6 convolution kernel) in each convolutional layer. Specifically, the formula for each sparse convolution is as follows:

[0048]

[0049] Among them, represents the number of non-empty positions in the input point cloud, and respectively represent the weight and kernel function of the sparse convolution, represents the kernel function centered on the i-th point in the input point cloud The spatial position information of the non-empty positions covered by, represents the data corresponding to this point after the sparse convolution operation.

[0050] After sparse convolution, the pooling layer downsamples the features extracted by the sparse convolutional layer, reducing the amount of data while retaining key features. Optionally, the number of pooling layers can be less than the number of sparse convolutional layers. Generally speaking, the more complex the shape of the ear mold, the fewer the number of pooling layers can be. The fully connected layer integrates the features extracted by the pooling layer to obtain a vector that can represent the local region features of the 3D ear sample.

[0051] The local region feature extraction network finally outputs the feature vector of the local region where each point in the point cloud data is located, such as a 512-dimensional feature vector, which reflects the local geometric characteristics of the outer ear. It should be noted that the local region here refers to a spatial range around a certain point, not the two types of regions to be recognized, and they should be distinguished.

[0052] After the two feature extraction networks complete feature extraction respectively, through an additional feature fusion matrix , the feature vectors of each point in the point cloud data and the features of the local area where the point is located are weighted and fused to obtain the final feature data of each point. Among them, a point can be understood as a matrix of the number of point clouds × 2, which is a learnable parameter matrix. The dimension 2 therein corresponds to the weights of the point features and the local features respectively. During the training process, the model learns whether each point in the point cloud depends more on the point information or the local information.

[0053] Finally, the final feature vector of each point is input into a multi-layer perceptron to classify each point into the sound outlet area and the non-sound outlet area. The final output of the model is , where , represents the judgment result of whether the i-th point in the 3D model of the hearing aid shell belongs to the sound outlet area.

[0054] S130. Perform spatial registration on the 3D model of the hearing aid shell and the 3D model of the outer ear to make the spatial positions of the two 3D models most similar.

[0055] In this step, spatial registration is performed on the 3D model of the hearing aid shell and the 3D model of the outer ear from which it is derived to maximize the spatial similarity of the two models. Figure 6 Fig. shows a registration result, in which the registered 3D model of the outer ear is marked with a red frame, and the registered 3D model of the hearing aid shell is marked with a black frame. It can be seen that when the spatial positions and the trending shapes of the same ear areas in the two models are closest (for example, the areas of one bend and two bends are in the same position and the shapes are basically close), the spatial registration is completed.

[0056] To achieve the above registration effect, this embodiment provides a reinforcement learning method. Using the features of the 3D model of the hearing aid shell and the 3D model of the outer ear as state variables, and using the spatial transformation from any one of the two 3D models to the other 3D model as an action variable, a reinforcement learning model is constructed; through reinforcement learning, the spatial transformation of the any one of the 3D models is gradually performed, and after each spatial transformation, with the increase of the spatial shape similarity of the two 3D models as the reward strategy, the action variable of the next spatial transformation is determined, and the spatial registration of the two 3D models is gradually realized.

[0057] In a specific embodiment, the point cloud PO (including n data points) of the 3D model of the outer ear can be expressed as:

[0058]

[0059] The point cloud PW (including m data points) of the 3D model of the hearing aid shell can be expressed as:

[0060]

[0061] Based on the above point cloud data, the state variables and action variables of the reinforcement learning model can be constructed.

[0062] Specifically, for the state variables, two sets of point cloud data can be transformed into global feature vectors through feature embedding, and the state variables can be generated using the two global feature vectors. Optionally, perform feature calculation on the three-dimensional model PO of the outer ear to construct a high-dimensional feature fo; perform feature calculation on the three-dimensional model PW of the hearing aid shell to construct a high-dimensional feature fw; then the state variable Status can be expressed as:

[0063] Statu = concat(fo, fw)

[0064] Among them, concat(,) represents the concatenation operation. Optionally, the feature calculation here can still adopt the network structure of the point feature extraction network in S120, and a max pooling layer is added after this structure. By taking the maximum value of all points for each feature dimension, the high-dimensional feature of the entire point cloud data can be obtained. Exemplarily, the dimensions of fo and fw can be taken as 1024.

[0065] At the same time, for the action variables, the spatial transformation from any one of the two three-dimensional models to the other three-dimensional model can be used as the action space, and the action space is divided to obtain each action variable.

[0066] Optionally, first perform direction division, and divide the three-dimensional space into a set of three directions Direct - Space = [X, Y, Z].

[0067] Then, according to the set unit translation distance, the translational action space along a single direction is divided into multiple discrete translational actions. Exemplarily, assuming that the minimum translation distance in the three-dimensional space is 0.05, then multiple discrete translation distances can be divided with this minimum translation distance as the unit, and the translational action space Lim_T can be expressed as:

[0068] Lim_T = [-0.2, -0.1, -0.05, 0, 0.05, 0.1, 0.2].

[0069] Similarly, according to the set unit rotation angle, the rotational action space along a single direction is divided into multiple discrete rotation angles, and the rotational action space Lim_R can be expressed as:

[0070] Lim_R = Lim_T × π / 2 = [-π / 10, -π / 20, -π / 40, 0, π / 40, π / 20, π / 10].

[0071] The above direction division, as well as translation and rotation divisions, can be simply represented as the direction space [0, 1, 2] and the scale space [0, 1, 2, 3, 4, 5, 6, 7]. Each combination of direction, translation scale, and rotation scale constitutes an action variable.

[0072] Furthermore, according to each action variable, an action function for spatial transformation from one 3D model to another 3D model in a two- or three-dimensional model can be constructed. This action function can be expressed as a combination of a translation function and a rotation function.

[0073] Among them, the translation function Translate(Direct, Lim) represents translation along a certain direction. The translation direction is the direction represented by the Direct-th element in the Direct-Space, and the translation distance is the distance corresponding to the Lim-th element in. The following will take the spatial transformation of PO as an example for illustration. Then, the translated point cloud PO1’ = PO + Translate(Direct, Lim).

[0074] The rotation function Rotate(Direct, LIM_R) represents rotation along a certain direction. The rotation direction is the direction represented by the Direct-th element in the Direct-Space, and the rotation angle is the angle corresponding to the Lim-th element in. The rotated point cloud PO2’ = Rotate(Direct,Lim)PO.

[0075] Furthermore, the two functions can be expressed in matrix form as:

[0076]

[0077]

[0078] So far, the construction of state variables and action variables has been completed. Then, continue to construct the reward function of the reinforcement learning model. Optionally, in this embodiment, the Chamfer distance between two point cloud sets is used to characterize the spatial similarity between two 3D models. Among them, the calculation formula of the Chamfer distance is:

[0079]

[0080] Among them, U and V are two point cloud sets to be compared, and u and v represent the points in U and V respectively, represents the number of points in the point cloud set U. The smaller the Chamfer distance between two point cloud sets, the higher the similarity. Based on the Chamfer distance, the following reward function can be constructed:

[0081] Reward = CD(PO, PW) - CD(PO’, PW)

[0082] Among them, Reward represents the reward value, PO’ represents PO after spatial transformation, CD(PO, PW) represents the Chamfer distance between PO before spatial transformation and PW, and CD(PO’, PW) represents the Chamfer distance between PO after spatial transformation and PW. Under this reward function, if the Chamfer distance between two 3D models becomes closer through spatial transformation, a positive reward is given to the current action; if the Chamfer distance between two 3D models becomes farther through spatial transformation, a negative reward is given to the current action, thus prompting the two 3D models to continuously approach in the direction of decreasing Chamfer distance.

[0083] Based on the above state variables, action variables, and reward function, reinforcement learning training can be carried out. Among them, the reinforcement learning model includes an Actor network and a decision network. The feature extraction network that generates the above state variable Status can be used as the Actor network, and Status is the output of the Actor network; the decision network can adopt a multi-layer perceptron structure or a Linear-based transformer network structure, which is used to evaluate each action variable according to Status and the reward function and determine the next action variable. During the training process, the reward value of each time is accumulated:

[0084] Rewards = SUM(CD(PO, PW) - CD(Dot(Rotate[Lim], PO) + Translate[Lim], PW))

[0085] Among them, Dot represents the dot product of matrices. When the cumulative reward value Rewards no longer increases, it can be determined that the two models are registered, that is, the effect shown in Figure 6 is achieved. The point cloud PWO after PO registration can be expressed as:

[0086] PWO = PO +

[0087] Among them, and represent the rotation matrix and translation matrix of the e-th step spatial transformation respectively.

[0088] It should be noted that the above S120 and S130 are independent of each other, and their order can be swapped, or they can be carried out simultaneously. This embodiment does not make specific restrictions.

[0089] S140. According to the positional relationship between the two registered 3D models, determine the second-bend cavity between the end face of the outer ear 3D model close to the eardrum and the sound outlet cavity.

[0090] The two - bend cavity to be recognized in this step is as shown Figure 7 in the red box of Figure 7 . Specifically, referring to

[0091] , after the registration of the two 3D models, the area between the end face close to the eardrum in the outer ear 3D model and the sound - outlet hole cavity of the hearing - aid shell is the two - bend cavity. This cavity extends along the two - bend direction of the ear canal and is very close to the direction of the user's eardrum, and can be used as the basis for subsequent judgment of the user's eardrum direction. i In a specific embodiment, the following operations can be respectively performed on each point U i in the registered outer ear 3D model: A triangle is formed by the center point wc1 of the registered hearing - aid shell 3D model, the center point wc2 of the registered sound - outlet hole cavity, and U Figure 8 as shown. If the angle with the center point wc2 of the sound - outlet hole cavity as the vertex in this triangle is an obtuse angle, it can be determined that the current U i belongs to the two - bend cavity.

[0092] Furthermore, the distance from wc1 to wc2 is denoted as L1, the distance from U i to wc1 is L2, and the distance from U i to wc2 is L3; if this triangle satisfies "this triangle is an obtuse - angled triangle and the side corresponding to L2 is the longest side of the triangle", then U i is considered to be a point on the two - bend cavity.

[0093] After performing the above - mentioned judgment on all U i , the point cloud of the two - bend cavity can be obtained.

[0094] S150. According to the orientations of the sound - outlet hole cavity and the two - bend cavity, judge the matching degree between the sound - outlet direction of the hearing - aid shell and the eardrum direction.

[0095] Through the operations of S110 - S140, two cavities are obtained in this embodiment. One is the sound - outlet hole cavity from the hearing - aid shell 3D model, and the other is the two - bend cavity from the outer ear 3D model. Among them, the axial direction of the sound - outlet hole cavity represents the sound - outlet direction of the hearing - aid shell, and the axial direction of the two - bend cavity area represents the direction of the user's eardrum. In this step, the axial directions of the two cavities will be accurately estimated, and the matching degree between the sound - outlet hole of the hearing - aid shell and the eardrum direction will be judged according to this axial direction.

[0096] In a specific embodiment, the two cavities can be processed respectively by the same method to obtain the axial directions of the two cavities respectively. Taking any one of the cavities as an example, this method includes the following steps:

[0097] Step 1: Use voxel downsampling to downsample the original point cloud of the cavity to be processed to reduce subsequent computational volume. Optionally, preset the voxel size p, and divide the three-dimensional space where the point cloud is located into several voxels according to this voxel size; then assign each point in the point cloud data to the corresponding voxel; for each voxel, take the coordinate average value of all points in the voxel as the representative point of the voxel, and the downsampled point cloud is composed of the representative points of all voxels.

[0098] Step 2: Determine the two end faces of the cavity according to the large curvature points of the downsampled point cloud. Optionally, for each point P in the downsampled point cloud i , calculate the curvature of this point respectively. The specific method is as follows: Determine a neighborhood with a radius of R centered on point P i , fit a plane according to the downsampled points in the neighborhood, and the plane equation is , where x, y, z are three-dimensional space coordinates, and a, b, c, d are the fitted plane parameters; then the normal vector of this plane can be expressed as (a, b, c), and the curvature k of P i can be calculated according to the following formula:

[0099]

[0100] Further, the process of plane fitting can be realized by the least squares method. In the least squares method, according to each point i in the neighborhood point set N , construct the objective function , and by taking the partial derivatives of a, b, c, d and setting the partial derivatives to 0, the fitting parameters a, b, c, d of the plane can be obtained.

[0101] After the curvature calculation is completed, according to the set curvature threshold, filter out the large curvature points in the cavity. These large curvature points are the points with relatively large morphological transitions in the cavity, concentrated at the intersection of the cavity side and the two ends of the cavity. Of course, for an irregular cavity, the cavity side will also include some large curvature points.

[0102] After the screening is completed, perform two-class classification on all large curvature points through spatial clustering. Optionally, the Kmeans clustering method can be used to cluster the spatial positions of the large curvature points, so that the large curvature points with similar spatial positions are clustered into one class, and finally two spatial position clustering clusters are obtained. Figure 9The blue cylinder simply illustrates a cavity, and the red and green boxes respectively illustrate the ranges of the two spatial position clusters. Furthermore, in the Kmeans clustering method, two initial centroids can be randomly selected from the large curvature points, and then each point is assigned to the nearest centroid, and the centroid of each cluster (i.e., the average position of all points in the cluster) is recalculated; each point is then reassigned to the nearest new centroid, and so on, until the final centroid no longer changes, and the final cluster and cluster center are obtained.

[0103] After clustering is completed, a set proportion of points that are farthest from the center point of the entire point cloud can be selected from the two clusters to form two new point cloud groups. For example, the point cloud composed of all large curvature points is recorded as B. For each cluster, the distance between each point in the cluster and the center point of B is calculated, and the distances are arranged from large to small. The points ranked in the top 1 / 3 are taken as a new point cloud group, thereby obtaining two new point cloud groups, such as Figure 10 Then, the center points of the two new point cloud groups are used as two new anchor points. and ; Calculation process and The straight line , and determine the passing point ,by is the plane F1 with normal direction, and the plane passing through point ,by If F2 is the plane in the normal direction, then F1 and F2 are the two end faces of the cavity.

[0104] Of course, the voxel down sampling in the above step 1 can also be omitted, and the process directly proceeds to step 2 to determine the two end faces of the cavity according to the large curvature points in the cavity. This embodiment is not specifically limited.

[0105] Step 3: Determine the preliminary axis of the two-bend cavity according to the center points of the two end faces; and determine the points on the side of the two-bend cavity according to the angle between the normal vector of each point in the two-bend cavity and the preliminary axis. Optionally, after obtaining the two end faces, the preliminary axis of the cavity can be determined according to the center points of the two end faces. Taking the above end faces F1 and F2 as an example, and can be regarded as their respective center points, It is the preliminary axis of the cavity. Points in the cavity located between the two end faces and whose normal vectors have an angle with the preliminary axis greater than a set threshold can be screened out. These points are points on the side of the cavity. For example, refer to Figure 11 , the blue arrows represent the normal vectors of each point in the cavity, then for each point T in the cavity point cloud i, if T i 's normal vector and 's included angle is greater than the set threshold (such as 65°), it indicates that T i is a point on the side of the cavity. Among them, the range of the included angle is [0, 90°]. For the calculation result of an included angle greater than 90°, subtract this result from 180°.

[0106] Step 4: Gradually move the points on the side of the cavity in the direction opposite to the normal vector of each point until the point density in the neighborhood of each point after moving decreases. Optionally, using the minimum moving distance in three-dimensional space as the step size, move each point on the cavity , respectively, in the direction opposite to the normal vector of (that is, shrink the points of the entire cavity radially inward to make it continuously approach the central axis of the cavity), and for each point , determine a neighborhood centered on . The neighborhood moves along with the movement before and after, but the neighborhood radius r remains unchanged. After moving all by one step size, complete one round of movement, and perform the following operations:

[0107] Compare the number of cavity points K1 and K2 in the neighborhood before and after moving each point ; if the number of cavity points K2 in the neighborhood after moving is less than the number K1 before moving, it indicates that this point has been over-moved and crossed the cavity axis; at this time, take a step back and do not change the position of in subsequent loops. After performing the above judgment on each point, enter the next round of movement, and so on, until the positions of all points no longer change. Finally, a new point set is formed, and this point set is very close to the axis of the cavity.

[0108] Step 5: According to the new point set , fit the axial straight line of the two-bend cavity. Optionally, the least squares method can be used to estimate the straight line for the point set . The point-direction equation of the straight line can be expressed as:

[0109]

[0110] Among them, x, y, and z are variables, , and t are parameters to be fitted, , and are the known data point coordinates.

[0111] In order to fit the above parameters, the following intermediate variables k1, b1, k2, and b2 can be constructed:

[0112]

[0113] Then the above-mentioned space straight line can be expressed as:

[0114]

[0115] Taking the coordinates of each point in the point set ([[]] , , ) respectively as , and , substituting them into the above equation, the sum of the squares of the residuals of these points and the two planes and can be obtained:

[0116]

[0117] Among them, Q represents the number of points in the point cloud . According to the least squares method, it is necessary to minimize and . Therefore, taking the derivatives of k1, b1, k2, and b2 respectively according to the above equation and setting the derivatives to 0, k1, b1, k2, and b2 can be solved, and then the parameters of the axial straight line equation of the double-bend cavity , and t can be calculated. The direction vector of this axial straight line in three-dimensional space is .

[0118] After performing the above operations from step 1 to step 5 on the double-bend cavity and the sound outlet hole cavity respectively, the axial straight line of the double-bend cavity can be obtained. This straight line is more accurate than the preliminary axis obtained previously and can better reflect the cavity trend.

[0119] Finally, according to the axial straight line of the double-bend cavity and the axial straight line of the sound outlet hole cavity, the matching degree between the sound outlet direction of the hearing aid shell and the eardrum direction is judged. Optionally, calculate the included angle of the two axial straight lines:

[0120]

[0121] Among them, represents the axial vector of the double-bend cavity; represents the axial vector of the sound outlet hole cavity. If the included angle is less than a certain threshold, it is determined that the sound outlet direction is normal; if the included angle is greater than or equal to this threshold, it is determined that the sound outlet direction is abnormal.

[0122] In summary, this embodiment provides a method for judging the matching degree between the sound outlet hole direction of a hearing aid shell and the eardrum direction. First, a three-dimensional region segmentation algorithm based on deep learning is used to identify the sound outlet hole region and the non-sound outlet hole region in the three-dimensional model of the hearing aid shell. Then, the three-dimensional model of the hearing aid shell and the three-dimensional model of the outer ear are spatially registered to make their spatial positions match. Next, the second-bend cavity in the three-dimensional model of the outer ear is identified according to the spatial positions of the two models after registration, as the basis for judging the axial direction of the user's eardrum. Then, the sound outlet hole cavity and the second-bend cavity are respectively shrunk in a point cloud radial shrinking manner to obtain point clouds that are almost parallel to the axes of the two cavities, and the axial lines of the two cavities are fitted according to these point clouds to ensure the accuracy of axial estimation. Finally, according to the axial angle between the sound outlet hole cavity and the second-bend cavity, it is accurately detected whether the sound outlet hole direction is qualified. Based on the three-dimensional model after modification by the mold modifier, the entire method can quickly realize the automatic detection of the sound outlet hole direction, with high accuracy and a speed that can be controlled within 150 ms, providing an important reference for the mold modifier to judge and adjust the sound outlet hole direction. At the same time, the preposition of the detection link greatly reduces the unnecessary resource waste in the subsequent process and fully guarantees the production quality of the hearing aid shell.

[0123] Figure 12 FIG. is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. As Figure 12 shown, the device includes a processor 60, a memory 61, an input device 62, and an output device 63. The number of processors 60 in the device can be one or more. Figure 12 Here, one processor 60 is taken as an example. The processor 60, the memory 61, the input device 62, and the output device 63 in the device can be connected through a bus or other means. Figure 12 Here, connection through a bus is taken as an example.

[0124] The memory 61, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as program instructions / modules corresponding to the method for judging the matching degree between the sound outlet hole direction of the hearing aid shell and the eardrum direction in the embodiment of the present invention. The processor 60 executes various functional applications and data processing of the device by running the software programs, instructions, and modules stored in the memory 61, that is, implements the above-mentioned method for judging the matching degree between the sound outlet hole direction of the hearing aid shell and the eardrum direction.

[0125] The memory 61 may mainly include a program storage area and a data storage area. Among them, the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 61 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some instances, the memory 61 may further include a memory remotely provided with respect to the processor 60, and these remote memories may be connected to the device through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0126] The input device 62 can be used to receive input digital or character information and generate key signal inputs related to the user settings and function controls of the device. The output device 63 may include a display device such as a display screen.

[0127] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the method for judging the matching degree between the sound outlet hole direction of the hearing aid housing and the eardrum direction in any embodiment.

[0128] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the computer-readable storage medium include: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device.

[0129] The computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries the computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, and this computer-readable medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device.

[0130] The program code contained on a computer-readable medium can be transmitted with any appropriate medium, including but not limited to wireless, wire, optical fiber cable, RF, etc., or any suitable combination of the above.

[0131] The computer program code for performing the operations of the present invention can be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as C language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0132] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the technical solutions of the embodiments of the present invention.

Claims

1. A method for judging the matching degree between the sound outlet direction of a hearing aid shell and the tympanic membrane direction, characterized in that Including: Obtain the three-dimensional model of the hearing aid shell to be judged and the three-dimensional model of the outer ear from which it is derived; Perform regional segmentation on the three-dimensional model of the hearing aid shell to identify the sound hole cavity; Perform spatial registration on the three-dimensional model of the hearing aid shell and the three-dimensional model of the outer ear to make the spatial positions of the two three-dimensional models most similar; According to the positional relationship between the two three-dimensional models after registration, determine the second-bend cavity between the end face of the outer ear three-dimensional model close to the eardrum and the sound hole cavity; Judge the matching degree between the sound hole direction of the hearing aid shell and the eardrum direction according to the trends of the sound hole cavity and the second-bend cavity.

2. The method according to claim 1, wherein The determining the second-bend cavity between the end face of the outer ear three-dimensional model close to the eardrum and the sound hole cavity according to the positional relationship between the two three-dimensional models after registration includes: Form triangles by the center of the three-dimensional model of the hearing aid shell after registration, the center of the sound hole cavity, and each point in the three-dimensional model of the outer ear after registration; If the angle with the center point of the sound hole cavity as the vertex in each triangle is an obtuse angle, judge that each point in the three-dimensional model of the outer ear after registration belongs to the second-bend cavity.

3. The method according to claim 1, characterized in that The judging the matching degree between the sound hole direction of the hearing aid shell and the eardrum direction according to the trends of the sound hole cavity and the second-bend cavity includes: Gradually move each point on the side surface of the second-bend cavity in the direction opposite to the normal vector of each point until the point density in the neighborhood of each point decreases after the movement; Fit the axial line of the second-bend cavity according to the position of each point before the last step of movement; Judge the matching degree between the sound hole direction of the hearing aid shell and the eardrum direction according to the axial line of the second-bend cavity and the axial line of the sound hole cavity.

4. The method according to claim 3, wherein Before the gradually moving each point on the side surface of the second-bend cavity in the direction opposite to the normal vector of each point, it further includes: Determine the two end faces of the second-bend cavity according to the large curvature points in the second-bend cavity; Determine the preliminary axis of the second-bend cavity according to the center points of the two end faces; Determine the points on the side surface of the second-bend cavity according to the included angle between the normal vector of each point in the second-bend cavity and the preliminary axis.

5. The method according to claim 4, characterized in that, The determining the two end faces of the second-bend cavity according to the large curvature points in the second-bend cavity includes: Screen the large curvature points in the second-bend cavity according to the set curvature threshold; Perform two-class classification on each large curvature point through spatial clustering; Respectively use the planes formed by the two clustering clusters as the two end faces of the second-bend cavity.

6. The method according to claim 5, characterized in that, The screening the large curvature points in the second-bend cavity according to the set curvature threshold includes: Perform voxel division on the second-bend cavity, and form a new point cloud of the second-bend cavity by each voxel; Screen large curvature points from the new point cloud according to the set curvature threshold.

7. The method according to claim 1, characterized in that, The performing regional segmentation on the three-dimensional model of the hearing aid shell to identify the sound hole cavity includes: Use a point feature extraction network based on deep learning to convert the spatial position information of each point in the three-dimensional model of the hearing aid shell into high-dimensional features of each point; Using a local region feature extraction network based on deep learning, extract the geometric features of the local regions from the three-dimensional model of the hearing aid housing to obtain the features of the local regions to which each point belongs; Using a feature fusion matrix, fuse and classify the high-dimensional features of each point and the features of the local region to which it belongs to determine whether each point belongs to the sound outlet cavity.

8. The method according to claim 1, wherein The spatial registration of the three-dimensional model of the hearing aid housing and the three-dimensional model of the outer ear to make the spatial positions of the two three-dimensional models most similar includes: Construct a reinforcement learning model with the high-dimensional features of the three-dimensional model of the hearing aid housing and the three-dimensional model of the outer ear as state variables and the spatial transformation from any one of the two three-dimensional models to the other three-dimensional model as action variables; Based on the reinforcement learning model, gradually perform spatial transformation on any one of the three-dimensional models, and after each spatial transformation, use the increase in the spatial position similarity of the two three-dimensional models as the reward strategy to decide the action variable of the next spatial transformation, and gradually realize the spatial registration of the two three-dimensional models.

9. An electronic device, characterized in that, Including: One or more processors; A memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method for judging the matching degree between the sound outlet direction of the hearing aid housing and the eardrum direction according to any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, Stored thereon is a computer program, which when executed by a processor implements the method for judging the matching degree between the sound outlet direction of the hearing aid housing and the eardrum direction according to any one of claims 1-8.

Images