Three-Dimensional Model Position Alignment Method for Customized Hearing Aids Based on Multi-Feature Fusion

Abstract

An embodiment of the present invention discloses a three-dimensional model position alignment method for a customized hearing aid based on multi-feature fusion, which relates to the technical field of three-dimensional modeling of hearing aids. Among them, the method includes: obtaining an operation model and a template model for customizing the ear mold of the hearing aid; aligning the centroids of the two models with the coordinate origin of the three-dimensional space; calculating the average curvature and local geometric structure information of each vertex in the two aligned models to obtain the local feature vector of each vertex; performing rough attitude adjustment and fine registration on the two models; performing position positioning on specific positions on the template model after registration; and respectively calculating the similarity between the feature vectors of the initially aligned vertex and multiple neighboring vertices around it and the feature vector of the vertex at the specific position, and identifying the vertex with the highest similarity as the final aligned vertex at the specific position. This embodiment realizes automatic and accurate identification of specific parts of the ear mold and efficient alignment.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of three-dimensional modeling of hearing aids, and in particular to a method for aligning the positions of three-dimensional models of customized hearing aids based on multi-feature fusion. Background Art

[0002] With the development of hearing aid technology, hearing aids are becoming more and more widely used. Current hearing aids are divided into standard types and customized types. Standard hearing aids are of the same size and are easy to process, but they are not suitable for users of all ages. Customized hearing aids are processed according to the individual ear mold models of specific users. Such hearing aid shells are comfortable to wear and are more popular.

[0003] There are many types of customized hearing aids, including in-the-ear and behind-the-ear hearing aids. Each type of hearing aid has a corresponding standard template. When processing the shell, the production personnel refer to the style of the standard template in the three-dimensional editing software, process the user's ear mold model into the appearance of the template, and then export the model data. The customized hearing aid shell can be printed through a 3D printer.

[0004] Although the structure of the ear is relatively consistent anatomically, due to significant differences in the length, curvature of the ear canal and the shape of the auricle of each person, it is necessary for the processing personnel to make personalized treatments according to their own experience. For example: when taking a reverse mold, there are a large number of redundant parts on the auricle, which need to be cut and deleted; or marks need to be made at the junction of the auricle and the ear canal, and holes need to be drilled at a certain position in the hearing aid. This requires determining the points on the user's ear mold model corresponding to certain specific positions on the template, so as to perform the same operations at the corresponding positions of the template. This process is very time-consuming and highly dependent on the personal experience and technical level of the processing personnel.

[0005] How to determine the corresponding positions of specific positions on the standard template on the user's ear mold model and achieve specific position alignment is an urgent problem to be solved. Summary of the Invention

[0006] The embodiments of the present invention provide a method for aligning the positions of three-dimensional models of customized hearing aids based on multi-feature fusion, which aligns the user's ear mold model with specific positions on the standard template, provides a basis for realizing personalized processing operations at specific positions, and improves the accuracy and processing speed of personalized processing of customized ear molds.

[0007] In a first aspect, the embodiments of the present invention provide a method for aligning the positions of three-dimensional models of customized hearing aids based on multi-feature fusion, including:

[0008] Obtain an operation model and a template model for customizing the ear mold of the hearing aid, wherein each model includes an ear canal part, an auricle part and a model bottom, and the model bottom is used to support the ear canal and auricle parts when the model is placed flat;

[0009] Align the centroids of the two models with the origin of the three-dimensional space coordinates;

[0010] Calculate the mean curvature and local geometric structure information of each vertex in the two aligned models, and arrange the calculation results as the local feature vectors of each vertex;

[0011] Convert the mean curvature of each vertex into RGB colors, and identify the largest same-color continuous area on the surface area of each aligned model as the bottom of the model; Centered on the centroid, rotate each of the two aligned models once so that their respective model bottoms are parallel to the x-y plane of the three-dimensional space; Centered on the centroid, rotate each of the two models after the first rotation around the z-axis for a second time so that the projection points of the highest points of the two models on the x-y plane and the origin of coordinates are on the same straight line;

[0012] Through minimizing the Chamfer distance, perform ear mold pose registration on the two models after the second rotation, and constrain the rotation angles of the operation model in each direction to be less than the set threshold during registration;

[0013] For specific positions on the registered template model, emit rays along the vertex normal directions at the specific positions, and determine the projection points of the rays on the registered operation model as the preliminary aligned vertices of the specific positions; Calculate the similarity between the feature vectors of the preliminary aligned vertices and multiple surrounding neighborhood vertices and the feature vectors of the vertices at the specific positions respectively, and identify the vertex with the highest similarity as the final aligned vertex of the specific position.

[0014] In a second aspect, an embodiment of the present invention provides an electronic device, and the electronic device includes:

[0015] One or more processors;

[0016] A memory for storing one or more programs,

[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the method for aligning the positions of the three-dimensional models of customized hearing aids based on multi-feature fusion according to any embodiment.

[0018] In a third aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the method for aligning the positions of the three-dimensional models of customized hearing aids based on multi-feature fusion according to any embodiment.

[0019] In summary, the embodiment of the present invention proposes a method for aligning the positions of three-dimensional models of customized hearing aids based on multi-feature fusion. By utilizing the morphological similarity of the ear mold and the multi-dimensional feature fusion representation ability, the exact positions corresponding to specific marked positions on the template ear are found on the operating ear, realizing automatic and accurate identification and efficient alignment of specific parts of the ear mold. Specifically, in this embodiment, grid reconstruction is first performed to make the number of grids of the two models basically the same and the topological forms basically consistent. Then, centroid translation is carried out. Through the recognition of the bottom of the model based on the curvature value color block, the alignment of the bottom plane of the model, and the collinearity of the projection point of the highest point and the origin direction, the ear canal directions of the models are adjusted to be consistent, realizing a rough adjustment of the ear mold posture. This method achieves a good effect of rough posture adjustment through simple geometric operations, improving the interpretability and computational efficiency of the entire method. Then, in this embodiment, fine registration of the ear mold posture is performed based on the minimization of the Chamfer distance. Compared with directly performing fine registration, after the rough posture adjustment, each vertex is very close to its registration object. Accurate registration can be achieved with very little computational effort, avoiding a large number of registration error points due to the far distance between corresponding point pairs in fine registration, and improving the alignment accuracy and computational efficiency. At the same time, by restricting the rotation of the operating model relative to each direction axis to be less than 5° in fine registration, unnecessary excessive calculations in fine registration are further avoided, further improving the alignment accuracy and computational efficiency. Finally, in this embodiment, preliminary vertex positioning is performed through the ray projection of specific positions on the operating model to lock the approximate area of the aligned vertices, and then local features are used to select the final aligned vertices from the approximate area, gradually improving the accuracy of the aligned vertices and finally achieving accurate position alignment. The entire method is simple to operate, highly interpretable, accurate in results, and high in execution efficiency, greatly improving the processing efficiency and product quality of hearing aids. Description of the Drawings

[0020] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the description of the specific embodiments or the prior art. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0021] Figure 1 is a flowchart of a method for aligning the positions of three-dimensional models of customized hearing aids based on multi-feature fusion provided by the embodiment of the present invention;

[0022] Figure 2 is a schematic diagram of an ear sample model provided by the embodiment of the present invention;

[0023] Figure 3 is a curvature value coloring diagram of an ear sample model provided by the embodiment of the present invention;

[0024] Figure 4 It is a schematic diagram of aligning the highest point of the ear canal provided by an embodiment of the present invention;

[0025] Figure 5 It is a flowchart of another method for aligning the positions of three-dimensional models of customized hearing aids based on multi-feature fusion provided by an embodiment of the present invention;

[0026] Figure 6 It is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. Detailed implementation manners

[0027] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below. Obviously, the described embodiments are only a part rather than all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the scope protected by the present invention.

[0028] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as a limitation to the present invention. In addition, the terms "first", "second", and "third" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance.

[0029] In the description of the present invention, it should also be noted that unless otherwise clearly defined and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0030] Figure 1 It is a flowchart of a method for aligning the positions of three-dimensional models of customized hearing aids based on multi-feature fusion provided by an embodiment of the present invention. This method can accurately locate the corresponding position on the user's ear impression model of a specific position on the template model, so as to perform the same operation at this position as at the specific position of the template model, improving the accuracy and processing speed of personalized processing of customized ear molds. This method is executed by an electronic device, such as Figure 1 shown, and the method specifically includes:

[0031] S110. Obtain an operation model and a template model for customizing a hearing aid ear impression.

[0032] Each model includes an ear canal part, an auricle part, and a model bottom, as Figure 2 shown. The ear canal part and the auricle part are respectively used to reflect the morphology of the ear canal and the auricle; the model bottom (i.e., Figure 2 the ear mold bottom in is used to support the entire model when the model is placed flat, so the bottom surface of the model bottom is usually relatively flat.

[0033] Furthermore, the operation model refers to the user's ear impression model, and the template model refers to the standard ear impression model of the hearing aid. In this embodiment, the operation model and the template model are first loaded into the same three-dimensional coordinate space as the data source of the entire method.

[0034] S120. Align the centroid of the two models with the coordinate origin of the three-dimensional space.

[0035] After the models are loaded, the surface mesh of the ear mold of the two models can be re-topologized first, so that the difference in the number of meshes of the two models after reconstruction is less than the set threshold. Optionally, tools such as QuadRemesher can be used to reconstruct the surface mesh of the operating ear (i.e., the operation model), and the surface is reconstructed into a uniform quadrilateral mesh. During the reconstruction process, relevant parameters are adjusted so that the number of surface meshes of the operation model and the template model is basically the same, so that the operation model and the template model have a unified topological structure. Optionally, to ensure that the model does not deform, the number of meshes should not be less than 10,000, and this value can vary according to actual needs.

[0036] After the mesh reconstruction, perform the centroid return to origin operation, and move the two reconstructed models respectively so that the centroid of the two models is aligned with the coordinate origin of the three-dimensional space. Optionally, first calculate the centroid coordinates of the three-dimensional ear mold, and the formula is as follows:

[0037]

[0038] where is the total number of vertices of the three-dimensional ear mold, is the coordinate of the i-th vertex, is the three-dimensional coordinate of the centroid. Subsequently, move the entire model so that the centroid of the two models coincides with the three-dimensional coordinate origin.

[0039] S130. Calculate the average curvature and local geometric structure information of each vertex in the two aligned models, and arrange each calculation result as the local feature vector of each vertex.

[0040] In this step, local features of the ear mold are recognized, four types of local features are calculated respectively, and feature vectors are constructed based on the calculation results, providing a basis for identifying the bottom of the ear model and distinguishing alignment points in subsequent operations.

[0041] In a specific embodiment, the following operations can be performed on the two aligned models respectively:

[0042] First, calculate the average curvature of each vertex , and this curvature is obtained by weighted averaging of the curvatures of the edges around the vertex and the angles between them. An exemplary formula is as follows:

[0043]

[0044] Where, represents the average curvature of the i -th vertex, represents the sum of the angles of the triangular faces around the i -th vertex, and is the neighborhood area around the i -th vertex.

[0045] Next, calculate the normalized normal vector of each vertex , that is, a vector perpendicular to the surface of the vertex neighborhood and with a modulus value equal to 1.

[0046] Secondly, using the principal component analysis method, calculate the local covariance matrix of the vertices and obtain the corresponding eigenvalues , and . Optionally, principal component analysis can be performed on the vector formed by arranging the coordinates of the vertex and its surrounding vertices, and the eigenvalues represent the high-dimensional implicit features of the vertex and its surrounding neighborhood in the mathematical space.

[0047] At the same time, use the FPFH (Fast Point Feature Histogram) algorithm to calculate the local geometric structure information of the vertices, such as: the normal vector angle , the relative distance of the neighborhood points , etc., where and represent different neighborhood points respectively.

[0048] Finally, arrange the average curvature, normalized normal vector, each eigenvalue, normal vector angle and relative distance of neighborhood points of each vertex in sequence to form the local feature vector of each vertex , where represent the values of each vector element respectively. This feature vector has rotational invariance and can effectively describe the local geometric shape.

[0049] S140. Convert the average curvature of each vertex into RGB colors, and identify the largest same-color continuous area in surface area among the aligned models as the bottom of the model; with the centroid as the center, rotate the two aligned models once respectively to make the bottom of their respective models parallel to the x-y plane of the three-dimensional space; with the centroid as the center, rotate the two models after the first rotation around the z-axis for a second time so that the projection points of the highest points of the two models on the x-y plane and the coordinate origin are on the same straight line.

[0050] This step performs the rough registration of the models, roughly adjusts the basic postures of the two models to be consistent, and provides a basis for the fine registration in subsequent operations.

[0051] In a specific implementation, first, the bottom of the model can be identified according to the average curvature. Since the bottom of the model is relatively flat and the average curvature is relatively consistent, in this embodiment, the vertices are colored according to the average curvature values to obtain a Figure 3 colored map as shown, where the largest continuous area of the same color in surface area (i.e., Figure 3 the largest green continuous area in

[0052] is the bottom of the model. After identification, the rough adjustment of the earmold posture is performed based on the bottom of the model. First, rotate the earmold once with the centroid as the center so that the bottom of the earmold is parallel to the x-y plane of the three-dimensional coordinate axes, and this plane is below the x-y plane (i.e., in the space where z < 0). Optionally, since the bottom of the earmold is relatively flat, any three points can be selected from the bottom contour lines of the two aligned models, and a plane is formed by these three points. When these two planes are rotated parallel to the x-y plane, it is considered that the bottoms of the two earmolds are parallel to the x-y plane.

[0053] Then, rotate the earmold around the z-axis with the centroid as the center until the ear canals of the operating ear and the template ear (i.e., the template model) are aligned in orientation. The specific judgment criteria are as follows: In the above steps, the bottom of the model has been rotated parallel to the xy plane and is in the space below the x-y plane. At this time, the ear canal part in the model faces upward, and the highest points of the two models are the highest points of the ear canals. Denote the projection point of the highest point of the ear canal of the operating model on the x-y plane as , and denote the projection point of the highest point of the ear canal of the template model on the x-y plane as . Rotate the two models around the z-axis. When , and the coordinate origin are on the same straight line, it is considered that the ear canals of the operating ear and the template ear are aligned in orientation. The screenshot of the final rotation result on the x-y plane is as Figure 4As shown, where the white area is the projection of the template model on the x-y plane, and the brown area is the projection of the operation model on the x-y plane. The red dots in the white area and the brown area are the projections of the highest points of the template model and the operation model on the x-y plane respectively.

[0054] S150. By minimizing the Chamfer distance, perform ear mold pose registration on the two models after the second rotation. During the registration, constrain the rotation angles of the operation model in each direction to be less than the set threshold.

[0055] This step performs fine registration of the ear mold pose. Specifically, S140 has performed preliminary pose registration on the operation model and the template model, and this step realizes fine ear mold pose registration by minimizing the Chamfer distance.

[0056] First, take the template ear and the operation ear as examples to illustrate the Chamfer distance. The Chamfer distance between the template ear and the operation ear can be expressed as:

[0057] (1)

[0058] Where, represents the vertex set of the template model, represents the vertex set of the operation model; represents the Chamfer distance between the two point sets, represents the vertex of the template model, represents the number of vertices of the template model, represents the vertex of the operation model, represents the number of vertices in the operation model.

[0059] Based on the above formula, the purpose of this step is to gradually solve a transformation matrix through an iterative calculation method, which can make the Chamfer distance between the two point sets the smallest. That is:

[0060] (2)

[0061] Where, represents the vertex set after rotating the operation model according to the rotation matrix T centered at the centroid. According to the above definition, only need to replace in formula (1) with , then the specific expression of can be obtained.

[0062] Meanwhile, during the iterative calculation process, the following constraints are added in this embodiment: operating on the ear The rotation angles in all directions must be less than a set threshold (such as 5°). At this time, since the operating model and the template model have maintained a relatively similar spatial posture after the rough registration in S140, only local rotation adjustment is required in this step to obtain an ideal registration state. By constraining the rotation angles along each coordinate axis to be less than, for example, 5°, unnecessary excessive calculations can be avoided, which may cause unnecessary registration errors. Optionally, after each iteration, check whether the rotation angle of the operating model around each coordinate axis relative to the initial operating model exceeds 5°; if the rotation angle around a certain coordinate axis exceeds 5°, stop rotating the operating model along the certain coordinate axis in the next iteration.

[0063] In this step, the best registration between the operating model and the template model is achieved by minimizing the Chamfer distance. The set of three-dimensional vertex coordinates of the operating model after registration is: .

[0064] S160. For specific positions on the registered template model, emit rays along the vertex normal at the specific positions, and determine the projection points of the rays on the registered operating model as the preliminary alignment vertices of the specific positions; calculate the similarities between the feature vectors of the preliminary alignment vertices and multiple surrounding neighborhood vertices and the feature vectors of the vertices at the specific positions respectively, and identify the vertex with the highest similarity as the final alignment vertex of the specific position.

[0065] The specific positions here refer to the special positions on the template model that require manual operations, such as the ear canal opening, the helix position, the cutting and punching positions, etc. This step locates the positions on the operating model for the specific positions on the template model.

[0066] In a specific embodiment, first, for the vertices at specific positions on the template ear , emit rays along the vertex normal. The projection points of the rays on the operating ear are the approximate alignment vertices of the operating ear with the template ear, or are called the preliminary alignment vertices. Then, take the local neighborhood points around the approximate alignment vertices, and compare the cosine similarities between the feature vectors of these vertices and the vertex feature vectors of. The vertex with the highest similarity is the final alignment vertex of the specific position on the operating model.

[0067] The above entire process can also be combined with Figure 5Understand the flowchart shown. Through six steps: mesh re-topology, centroid back to the origin, local feature recognition, rough pose adjustment, fine pose adjustment, and position alignment, the three-dimensional model position of the customized hearing aid is automatically aligned, eliminating the manual recognition and alignment process, and improving the accuracy and processing speed of the personalized processing of the customized ear mold.

[0068] In summary, this embodiment proposes a method for aligning the three-dimensional model position of a customized hearing aid based on multi-feature fusion. By utilizing the morphological similarity of the ear mold and the multi-dimensional feature fusion representation ability, the exact position corresponding to a specific marked position on the template ear is found on the operating ear, realizing automatic and accurate identification and efficient alignment of specific parts of the ear mold. Specifically, in this embodiment, mesh reconstruction is first performed to make the number of meshes of the two models basically the same and the topological forms basically consistent. Then, centroid translation is carried out. Through the recognition of the bottom of the model based on the curvature value color block, the alignment of the bottom plane of the model, and the collinearity of the projection point of the highest point and the origin direction, the ear canal directions of the models are adjusted to be consistent, realizing the rough pose adjustment of the ear mold. This method achieves a good rough pose adjustment effect through simple geometric operations, improving the interpretability and computational efficiency of the entire method. Then, in this embodiment, fine registration of the ear mold pose is performed based on the minimization of the Chamfer distance. Compared with directly performing fine registration, after rough pose adjustment, each vertex is very close to its registration object. Accurate registration can be achieved with very little computational effort, avoiding a large number of registration error points due to the far distance between corresponding point pairs in fine registration, and improving the alignment accuracy and computational efficiency. At the same time, by restricting the rotation of the operating model relative to each axis to be less than 5° in fine registration, unnecessary over-computation in fine registration is further avoided, further improving the alignment accuracy and computational efficiency. Finally, in this embodiment, preliminary vertex positioning is performed through the ray projection of a specific position on the operating model to lock the approximate area of the alignment vertex, and then local features are used to select the final alignment vertex from the approximate area, gradually improving the accuracy of the alignment vertex and finally achieving accurate position alignment. The entire method is simple to operate, highly interpretable, accurate in results, and efficient in execution, greatly improving the processing efficiency and product quality of the hearing aid.

[0069] Figure 6 The following is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. As Figure 6 shown, the device includes a processor 60, a memory 61, an input device 62, and an output device 63; the number of processors 60 in the device can be one or more, Figure 6 taking one processor 60 as an example; the processor 60, memory 61, input device 62, and output device 63 in the device can be connected through a bus or other means, Figure 6 taking connection through a bus as an example.

[0070] The memory 61, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as program instructions / modules corresponding to the method for aligning the positions of three-dimensional models of customized hearing aids based on multi-feature fusion in the embodiments of the present invention. The processor 60 executes various functional applications and data processing of the device by running the software programs, instructions, and modules stored in the memory 61, that is, to implement the above-mentioned method for aligning the positions of three-dimensional models of customized hearing aids based on multi-feature fusion.

[0071] The memory 61 may mainly include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the terminal, etc. In addition, the memory 61 may include high-speed random access memory and may also include non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some instances, the memory 61 may further include a memory remotely set relative to the processor 60, and these remote memories can be connected to the device through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0072] The input device 62 can be used to receive input digital or character information, and generate key signal inputs related to the user settings and function controls of the device. The output device 63 may include a display device such as a display screen.

[0073] The embodiments of the present invention also provide a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the method for aligning the positions of three-dimensional models of customized hearing aids based on multi-feature fusion in any embodiment.

[0074] The computer storage medium of the embodiments of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the computer-readable storage medium include: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer-readable storage medium can be any tangible medium that contains or stores a program, and the program can be used by or combined with an instruction execution system, apparatus, or device.

[0075] A computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device.

[0076] The program code contained on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0077] Computer program code for carrying out operations of the present invention may be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also conventional procedural programming languages such as the C language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0078] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the technical solutions of the embodiments of the present invention.

Claims

1. A method for aligning the position of a three-dimensional model of a customized hearing aid based on multi-feature fusion, characterized in that: include: Obtaining an operating model and a template model for customizing an ear sample for a hearing aid, wherein each model includes an ear canal portion, an auricle portion, and a model bottom, wherein the model bottom is used to support the ear canal and the auricle portion when the model is laid flat; Align the centroids of the two models with the coordinate origin of the three-dimensional space; Calculate the average curvature and local geometric structure information of each vertex in the two aligned models, and arrange the calculation results into a local feature vector of each vertex; The average curvature of each vertex is converted into RGB color, and the continuous area of ​​the same color with the largest surface area in each model after alignment is identified as the bottom of the model; the two aligned models are rotated once with the center of mass as the center, so that the bottom of each model is parallel to the xy plane of the three-dimensional space; the two models after the rotation are rotated twice around the z axis with the center of mass as the center, so that the projection point of the highest point of the two models on the xy plane is on the same straight line with the origin of the coordinate system; By minimizing the Chamfer distance, the two models after the secondary rotation are aligned with the ear mold posture, and the rotation angle of the operation model in each direction is constrained to be less than a set threshold during the alignment; For a specific position on the registered template model, a ray is emitted along the vertex normal at the specific position, and the projection point of the ray on the registered operation model is determined as the preliminary alignment vertex of the specific position; the similarity between the local feature vectors of the preliminary alignment vertex and multiple neighborhood vertices around it and the local feature vector of the vertex at the specific position is calculated respectively, and the vertex with the highest similarity is identified as the final alignment vertex of the specific position.

2. The method according to claim 1, characterized in that The step of aligning the centroids of the two models with the coordinate origin of the three-dimensional space includes: Reconstruct the surface meshes of the two models respectively so that the difference in the number of meshes between the two models is less than the set threshold; The two reconstructed models are moved separately so that the centroids of the two models are aligned with the coordinate origin of the three-dimensional space.

3. The method according to claim 2, characterized in that Before the two reconstructed models are moved respectively to align the centroids of the two models with the coordinate origin of the three-dimensional space, the method further includes: The centroid of any model is calculated according to the following formula: Where N is the total number of vertices of any model, P i is the three-dimensional coordinate of the i-th vertex, and C is the three-dimensional coordinate of the center of mass.

4. The method according to claim 1, characterized in that The calculation is performed to align the average curvature and local geometric structure information of each vertex in the two models, and the calculation results are arranged into a local feature vector of each vertex, including: For the two aligned models, calculate the average curvature and normalized normal vector of each vertex respectively; The principal component analysis method is used to calculate the local covariance matrix and multiple eigenvalues ​​of each vertex. Specifically, the principal component analysis is performed on the vector formed by the coordinate arrangement of the vertex and the surrounding vertices. The eigenvalue represents the high-dimensional implicit characteristics of the vertex and its surrounding neighborhood in the mathematical space. Use the FPFH algorithm to calculate the normal vector angle of each vertex and the relative distance of the neighboring points; The average curvature, normalized normal vector, eigenvalues, normal vector angle and relative distance of neighboring points of each vertex are arranged in sequence to form the local eigenvector of each vertex.

5. The method according to claim 4, characterized in that The step of calculating the average curvature of each vertex includes: Calculate the average curvature of each vertex according to the following formula: Among them, K i represents the mean curvature of the ith vertex, ∑ 邻边 θ i represents the sum of the angles of the triangles around the i-th vertex, and A is the neighborhood area around the i-th vertex.

6. The method according to claim 1, characterized in that The step of rotating the two aligned models with the center of mass as the center so that the bottom of each model is parallel to the xy plane of the three-dimensional space includes: Select three points at random from the flat bottom contours of the two aligned models to form two planes respectively; The two aligned models are rotated once with the center of mass as the center, so that the two planes are parallel to the xy plane of the three-dimensional space and are located below the xy plane.

7. The method according to claim 1, characterized in that The method of performing ear mold posture registration on the two models after the secondary rotation by minimizing the Chamfer distance, wherein the rotation angle of the operation model in each direction is constrained to be less than a set threshold, comprises: Iteratively solve the rotation matrix T centered on the center of mass so that T = arg min T D Chamfer (P tem, T.P op ), where P tem Represents the vertex set of the template model, P op Represents the vertex set of the operation model, T·P op Indicates the vertex set after the operation model is rotated according to the rotation matrix T; D Chamfer (,) represents the Chamfer distance between two point sets, p1 represents the vertex of the template model, |p1| represents the number of vertices of the template model, and p2 represents T·P op The vertex in T·P op The number of vertices in ; After each iteration, check T·P op Relative to P op Whether the rotation angle around each coordinate axis exceeds the set threshold; if the rotation angle around any coordinate axis exceeds the set threshold, stop rotating along any coordinate axis in the next iteration op .

8. The method according to claim 1, characterized in that The respectively calculating the similarity between the feature vectors of the preliminary aligned vertex and a plurality of neighboring vertices around it and the feature vector of the vertex at the specific position includes: Calculating the cosine similarity between the feature vector of the preliminary aligned vertex and the feature vector of the vertex at the specific position; The cosine similarities between the feature vectors of a plurality of neighborhood vertices around the preliminary aligned vertex and the feature vector of the vertex at the specific position are calculated respectively.

9. An electronic device, characterized in that: include: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method for position alignment of a customized hearing aid three-dimensional model based on multi-feature fusion as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the program is executed by a processor, the method for position alignment of a three-dimensional model of a customized hearing aid based on multi-feature fusion as described in any one of claims 1 to 8 is implemented.

Images