Customized hearing aid three-dimensional model position alignment method based on multi-feature fusion

Abstract

The embodiment of the invention discloses a customized hearing aid three-dimensional model position alignment method based on multi-feature fusion, and relates to the technical field of hearing aid three-dimensional modeling. The method comprises the following steps: acquiring an operation model and a template model for customizing a hearing aid ear sample; aligning the centroids of the two models with the origin of coordinates of the three-dimensional space; calculating an average curvature and local geometric structure information of each vertex in the two aligned models to obtain a local feature vector of each vertex; carrying out attitude coarse and fine registration on the two models; carrying out position positioning on a specific position on the template model after registration; and respectively calculating the similarity between the feature vectors of the preliminarily aligned vertex and a plurality of neighborhood vertexes around the preliminarily aligned vertex and the feature vector of the vertex at the specific position, and identifying the vertex with the highest similarity as a final aligned vertex at the specific position. According to the embodiment of the invention, automatic and accurate identification and efficient alignment of the specific part of the ear model are realized.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of hearing aid three-dimensional modeling, and in particular to a method for aligning the position of a customized hearing aid three-dimensional model based on multi-feature fusion. Background Art

[0002] With the development of hearing aid technology, hearing aids are used more and more widely. Currently, hearing aids are divided into standard and customized. Standard hearing aids are of the same size and simple to process, but they are not suitable for users of all ages. Customized hearing aids are processed according to the individual ear model of the specific user. This type of hearing aid shell is comfortable to wear and is more popular.

[0003] There are many types of custom hearing aids, including in-the-ear and behind-the-ear hearing aids. Each type of hearing aid has a corresponding standard template. When processing the shell, the production staff refers to the style of the standard template in the 3D editing software, processes the user's ear model into the template, and then exports the model data, and prints the customized hearing aid shell through a 3D printer.

[0004] Although the structure of the ear is relatively consistent in anatomy, the length, curvature and shape of the ear canal of each person are significantly different, so the processor needs to make personalized processing based on his or her own experience. For example, when casting, there are a lot of redundant parts in the auricle that need to be cut and deleted; or the hearing aid needs to mark the junction of the auricle and the ear canal, or punch a hole in a certain part. This requires determining the corresponding points in the user's ear model at certain specific positions in the template, so as to perform the same operations as the corresponding positions in the template. This process is very time-consuming and highly dependent on the personal experience and technical level of the processor.

[0005] How to determine the corresponding position of a specific position on the standard template on the user's ear sample model and achieve alignment of the specific position is an urgent problem to be solved. Summary of the invention

[0006] An embodiment of the present invention provides a method for aligning the position of a customized hearing aid three-dimensional model based on multi-feature fusion, which aligns the user's ear sample model with a specific position on a standard template, provides a basis for realizing personalized processing operations at specific positions, and improves the accuracy and speed of personalized processing of customized ear molds.

[0007] In a first aspect, an embodiment of the present invention provides a method for aligning a three-dimensional model of a customized hearing aid based on multi-feature fusion, comprising: Obtaining an operating model and a template model for customizing an ear sample for a hearing aid, wherein each model includes an ear canal portion, an auricle portion, and a model bottom, wherein the model bottom is used to support the ear canal and the auricle portion when the model is laid flat; Align the centroids of the two models with the coordinate origin of the three-dimensional space; Calculate the average curvature and local geometric structure information of each vertex in the two aligned models, and arrange the calculation results into local feature vectors of each vertex; The average curvature of each vertex is converted into RGB color, and the continuous area of ​​the same color with the largest surface area in each model after alignment is identified as the bottom of the model; the two aligned models are rotated once with the center of mass as the center, so that the bottom of each model is parallel to the xy plane of the three-dimensional space; the two models after the rotation are rotated twice around the z axis with the center of mass as the center, so that the projection point of the highest point of the two models on the xy plane is on the same straight line with the origin of the coordinate system; By minimizing the Chamfer distance, the two models after the secondary rotation are aligned with the ear mold posture, and the rotation angle of the operation model in each direction is constrained to be less than a set threshold during the alignment; For a specific position on the registered template model, a ray is emitted along the vertex normal at the specific position, and the projection point of the ray on the registered operation model is determined as the preliminary alignment vertex of the specific position; the similarity between the feature vectors of the preliminary alignment vertex and multiple neighboring vertices around it and the feature vector of the vertex at the specific position is calculated respectively, and the vertex with the highest similarity is identified as the final alignment vertex of the specific position.

[0008] In a second aspect, an embodiment of the present invention provides an electronic device, the electronic device comprising: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method for position alignment of a customized hearing aid three-dimensional model based on multi-feature fusion as described in any embodiment.

[0009] In a third aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for position alignment of a three-dimensional model of a customized hearing aid based on multi-feature fusion as described in any embodiment.

[0010] In summary, the embodiment of the present invention proposes a method for aligning the position of a three-dimensional model of a customized hearing aid based on multi-feature fusion. By utilizing the morphological similarity of the ear mold and the multi-dimensional feature fusion representation capability, the precise position corresponding to the specific mark position on the template ear on the operating ear is found, thereby realizing automatic and accurate identification and efficient alignment of specific parts of the ear mold. Specifically, the present embodiment first performs mesh reconstruction so that the number of meshes of the two models is basically the same and the topological morphology is basically consistent; then the centroid translation is performed, and the ear canal direction of the model is adjusted to be consistent through the identification of the bottom of the model based on the curvature value color block, as well as the alignment of the bottom plane of the model and the collinearity of the highest point projection point with the origin, so as to achieve a rough adjustment of the ear mold posture. This method achieves a good rough adjustment effect of the posture through simple geometric operations, and improves the interpretability and computational efficiency of the entire method; then, the present embodiment performs fine alignment of the ear mold posture based on the minimization of the Chamfer distance. Compared with direct fine alignment, each vertex after the rough adjustment of the posture is very close to its own alignment object, and through very A small amount of calculation can achieve accurate registration, avoid a large number of registration error points due to the long distance between corresponding points in fine registration, and improve the alignment accuracy and calculation efficiency; at the same time, by constraining the rotation of the operation model relative to each direction axis to be less than 5° in fine registration, it further avoids unnecessary over-calculation in fine registration, and further improves the alignment accuracy and calculation efficiency; finally, this embodiment performs preliminary vertex positioning through ray projection of specific positions on the operation model, locks the approximate area of ​​the alignment vertex, and then uses local features to select the final alignment vertex from the approximate area, thereby gradually improving the accuracy of the alignment vertex and finally achieving accurate position alignment. The whole method is simple to operate, highly interpretable, accurate in results, and highly efficient in execution, which greatly improves the processing efficiency and product quality of hearing aids. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0012] Figure 1 is a flow chart of a method for aligning a three-dimensional model of a customized hearing aid based on multi-feature fusion provided by an embodiment of the present invention; Figure 2 is a schematic diagram of an ear-like model provided by an embodiment of the present invention; Figure 3 is a curvature value colored graph of an ear-like model provided by an embodiment of the present invention; Figure 4is a schematic diagram of aligning the highest points of an ear canal provided by an embodiment of the present invention; Figure 5 is a flow chart of another method for aligning a three-dimensional model of a customized hearing aid based on multi-feature fusion provided by an embodiment of the present invention; Figure 6 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0013] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.

[0014] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0015] In the description of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0016] Figure 1 The flowchart of a method for aligning the position of a three-dimensional model of a customized hearing aid based on multi-feature fusion provided by an embodiment of the present invention is shown. The method can accurately locate the position corresponding to a specific position on the template model on the user's ear sample model, so as to perform the same operation at the position as that at the specific position of the template model, thereby improving the accuracy and processing speed of the personalized processing of the customized ear mold. The method is executed by an electronic device, such as Figure 1 As shown, the method specifically includes: S110, obtaining an operation model and a template model for customizing a hearing aid ear sample.

[0017] Each model includes an ear canal part, an auricle part and a model bottom, such as Figure 2 The ear canal part and the auricle part are used to reflect the morphology of the ear canal and auricle respectively; the bottom of the model (i.e. Figure 2 The bottom of the ear mold in the figure is used to support the entire model when it is laid flat, so the bottom surface of the model is usually relatively flat.

[0018] Furthermore, the operation model refers to the ear sample model of the user, and the template model refers to the standard ear sample model of the hearing aid. In this embodiment, the operation model and the template model are first loaded into the same three-dimensional coordinate space as the data source of the entire method.

[0019] S120, aligning the centroids of the two models with the coordinate origin of the three-dimensional space.

[0020] After the model is loaded, the surface mesh of the ear molds of the two models can be retopologically modified so that the difference in the number of meshes of the two models after reconstruction is less than the set threshold. Optionally, the surface mesh of the operating ear (i.e., the operating model) can be reconstructed using tools such as QuadRemesher to reconstruct the surface into a uniform quadrilateral mesh. During the reconstruction process, relevant parameters are adjusted so that the number of surface meshes of the operating model and the template model is basically the same, so that the operating model and the template model have a unified topological structure. Optionally, to ensure that the model does not deform, the number of meshes should not be less than 10,000, and this value can be changed according to actual needs.

[0021] After the mesh is reconstructed, the centroid return operation is performed to move the two reconstructed models respectively so that the centroids of the two models are aligned with the coordinate origin of the three-dimensional space. Optionally, the centroid coordinates of the three-dimensional ear mold are first calculated, and the formula is as follows:

[0022] in, is the total number of vertices of the three-dimensional ear mold, is the coordinate of the ith vertex, is the 3D coordinate of the center of mass. Then move the entire model so that the center of mass of the two models coincides with the origin of the 3D coordinate system.

[0023] S130, calculating the average curvature and local geometric structure information of each vertex in the two aligned models, and arranging the calculation results into a local feature vector of each vertex.

[0024] This step performs local feature recognition of the ear mold, calculates four types of local features respectively, and constructs feature vectors based on the calculation results, providing a basis for identifying the bottom of the ear model and distinguishing alignment points in subsequent operations.

[0025] In a specific implementation, the following operations may be performed on the two aligned models respectively: First, calculate the mean curvature of each vertex , which is obtained by weighted average of the curvature of the edges around the vertex and the angle between them. An exemplary formula is as follows:

[0026] in, Indicates i The average curvature of the vertices, Indicates i The sum of the angles of the triangles around the vertex, It is i The neighborhood area around a vertex.

[0027] Next, calculate the normalized normal vector for each vertex , that is, a vector perpendicular to the vertex neighborhood surface with a modulus equal to 1.

[0028] Secondly, the principal component analysis method is used to calculate the local covariance matrix of the vertex and obtain the corresponding eigenvalues , and Optionally, principal component analysis can be performed on the vector formed by arranging the coordinates of the vertex and the surrounding vertices. The eigenvalue represents the high-dimensional implicit features of the vertex and its surrounding neighborhood in the mathematical space.

[0029] At the same time, the FPFH (Fast Point Feature Histogram) algorithm is used to calculate the local geometric structure information of the vertex, such as the normal vector angle , the relative distance of the neighboring points etc., among which, and They represent different neighborhood points respectively.

[0030] Finally, the average curvature, normalized normal vector, eigenvalues, normal vector angle and relative distance of neighboring points of each vertex are arranged in sequence to form the local eigenvector of each vertex. ,in, Represent the values ​​of each vector element. The eigenvector is rotationally invariant and can effectively describe the local geometric shape.

[0031] S140, converting the average curvature of each vertex into RGB color, identifying the continuous area of ​​the same color with the largest surface area in each model after alignment as the bottom of the model; rotating the two aligned models once with the centroid as the center, so that the bottom of each model is parallel to the xy plane of the three-dimensional space; rotating the two models once around the z-axis for a second time with the centroid as the center, so that the projection point of the highest point of the two models on the xy plane is on the same straight line with the origin of the coordinate system.

[0032] This step performs a rough alignment of the models, roughly adjusting the basic postures of the two models to be consistent, providing a basis for fine alignment in subsequent operations.

[0033] In a specific implementation, first, the bottom of the model can be identified based on the average curvature. Since the bottom of the model is relatively flat and the average curvature is relatively consistent, this embodiment colors the vertices according to the average curvature value, and obtains the following: Figure 3 The colored graph shown in the figure is the one in which the continuous region of the same color with the largest surface area (i.e. Figure 3 The largest green continuous area in the figure is the bottom of the model.

[0034] After the recognition is completed, the ear mold posture is roughly adjusted based on the bottom of the model. First, the ear mold is rotated once with the center of mass as the center, so that the bottom of the ear mold is parallel to the three-dimensional coordinate axis xy plane, and the plane is located below the xy plane (that is, in the space where z<0). Optionally, since the bottom of the ear mold is relatively flat, three points can be randomly selected from the bottom contour lines of the two models after alignment, and a plane is formed by these three points. When these two planes are rotated parallel to the xy plane, it is considered that the bottoms of the two ear molds are parallel to the xy plane.

[0035] Next, rotate the ear mold around the z-axis with the center of mass as the center until the ear canal of the operating ear is aligned with the ear canal of the template ear (i.e., the template model). The specific judgment criteria are as follows: In the above steps, the bottom of the model has been rotated parallel to the xy plane and is located in the space below the xy plane. At this time, the ear canal part of the model is facing upward, and the highest point of the two models is the highest point of the ear canal. The projection point of the highest point of the ear canal of the operating model on the xy plane is recorded as , the projection point of the highest point of the ear canal of the template model on the xy plane is recorded as , rotate the two models around the z axis, when , When the three points of the coordinate origin are on the same straight line, the ear canal of the operating ear is considered to be aligned with the ear canal of the template ear. The screenshot of the final rotation result on the xy plane is as follows: Figure 4 As shown, the white area is the projection of the template model on the xy plane, the brown area is the projection of the operation model on the xy plane, and the red dots in the white area and the brown area are the projections of the highest points of the template model and the operation model on the xy plane, respectively.

[0036] S150, performing ear mold posture alignment on the two models after the secondary rotation by minimizing the Chamfer distance, and constraining the rotation angle of the operation model in each direction to be less than a set threshold during the alignment.

[0037] This step performs fine registration of the ear mold posture. Specifically, S140 performs preliminary registration of the operation model and the template model, and this step implements fine registration of the ear mold posture by minimizing the Chamfer distance.

[0038] First, use the template ear and operate the ears As an example, the Chamfer distance is explained. and operate the ears The Chamfer distance can be expressed as: (1) in, Represents the vertex set of the template model, A vertex set representing the operation model; represents the Chamfer distance between two point sets, Represents the vertices of the template model, Indicates the number of vertices of the template model, Represents the vertices of the operation model, Indicates the number of vertices in the operation model.

[0039] Based on the above formula, this step aims to gradually solve a transformation matrix through iterative calculation method: , which can make The Chamfer distance between two point sets is the smallest. That is: (2) in, Indicates that the model will be operated according to the rotation matrix centered on the center of mass T The set of vertices after rotation. According to the above definition, we only need to replace Replace with , you can get specific expression of.

[0040] At the same time, in the iterative calculation process, this embodiment adds the following constraints: The rotation angle along each direction must be less than the set threshold (e.g. 5°). At this time, since the operation model and the template model have maintained a relatively similar spatial posture after the rough registration in S140, only local rotation adjustment is required in this step to obtain a relatively ideal registration state. By constraining the rotation angle along each coordinate axis to be less than, for example, 5°, unnecessary excessive calculations can be avoided, which may cause unnecessary registration errors. Optionally, after each iteration, check Whether the rotation angle around each coordinate axis relative to the initial operation model exceeds 5°; if the rotation angle around a certain coordinate axis exceeds 5°, stop rotating the operation model along the certain coordinate axis in the next iteration.

[0041] This step achieves the best registration of the operation model and the template model by minimizing the Chamfer distance. The three-dimensional vertex coordinate set of the operation model after registration is: .

[0042] S160. For a specific position on the registered template model, emit a ray along the vertex normal at the specific position, and determine the projection point of the ray on the registered operation model as the preliminary alignment vertex of the specific position; calculate the similarity between the feature vectors of the preliminary alignment vertex and multiple neighboring vertices around it and the feature vector of the vertex at the specific position, and identify the vertex with the highest similarity as the final alignment vertex at the specific position.

[0043] The specific position here refers to the special position in the template model that needs to be manually operated, such as the ear canal opening, the helix position, the cutting and punching position, etc. This step locates the specific position on the template model on the operation model.

[0044] In a specific embodiment, first, a vertex at a specific position on the template ear is , a ray is emitted along the vertex normal, and the projection point of the ray on the operating ear This is the approximate alignment vertex between the operating ear and the template ear, or the preliminary alignment vertex. Next, take the approximate alignment vertex around the vertex. local neighborhood points, and the feature vectors of these vertices and The vertex feature vectors of are compared by cosine similarity, and the vertex with the highest similarity is the final aligned vertex at the specific position on the operation model.

[0045] The above process can also be combined with Figure 5 The flowchart shown can be understood. Through the six steps of mesh retopology, center of mass return to the origin, local feature recognition, posture coarse adjustment, posture fine adjustment and position alignment, the position alignment of the customized hearing aid three-dimensional model is automatically realized, eliminating the process of manual identification and alignment, and improving the accuracy and speed of personalized processing of customized ear molds.

[0046] In summary, this embodiment proposes a method for aligning the position of a three-dimensional model of a customized hearing aid based on multi-feature fusion. By utilizing the morphological similarity of the ear mold and the multi-dimensional feature fusion representation capability, the precise position corresponding to the specific mark position on the template ear on the operating ear is found, thereby realizing automatic and accurate identification and efficient alignment of specific parts of the ear mold. Specifically, this embodiment first performs mesh reconstruction so that the number of meshes of the two models is basically the same and the topological morphology is basically consistent; then the centroid translation is performed, and the ear canal direction of the model is adjusted to be consistent through the identification of the bottom of the model based on the curvature value color block, as well as the alignment of the bottom plane of the model and the collinearity of the highest point projection point with the origin, so as to achieve a rough adjustment of the ear mold posture. This method achieves a good rough adjustment effect of the posture through simple geometric operations, and improves the interpretability and computational efficiency of the entire method; then, this embodiment performs fine alignment of the ear mold posture based on the minimization of the Chamfer distance. Compared with direct fine alignment, each vertex after the rough adjustment of the posture is very close to its own alignment object, and through very A small amount of calculation can achieve accurate registration, avoid a large number of registration error points due to the long distance between corresponding points in fine registration, and improve the alignment accuracy and calculation efficiency; at the same time, by constraining the rotation of the operation model relative to each direction axis to be less than 5° in fine registration, it further avoids unnecessary over-calculation in fine registration, and further improves the alignment accuracy and calculation efficiency; finally, this embodiment performs preliminary vertex positioning through ray projection of specific positions on the operation model, locks the approximate area of ​​the alignment vertex, and then uses local features to select the final alignment vertex from the approximate area, thereby gradually improving the accuracy of the alignment vertex and finally achieving accurate position alignment. The whole method is simple to operate, highly interpretable, accurate in results, and highly efficient in execution, which greatly improves the processing efficiency and product quality of hearing aids.

[0047] Figure 6 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention is shown in FIG. Figure 6 As shown, the device includes a processor 60, a memory 61, an input device 62 and an output device 63; the number of processors 60 in the device can be one or more. Figure 6 A processor 60 is taken as an example; the processor 60, the memory 61, the input device 62 and the output device 63 in the device can be connected by a bus or other means. Figure 6 The example of connecting through bus is taken in the following.

[0048] The memory 61, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the method for aligning the position of a three-dimensional model of a customized hearing aid based on multi-feature fusion in the embodiment of the present invention. The processor 60 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 61, that is, realizing the above-mentioned method for aligning the position of a three-dimensional model of a customized hearing aid based on multi-feature fusion.

[0049] The memory 61 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 61 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 61 may further include a memory remotely arranged relative to the processor 60, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0050] The input device 62 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the device. The output device 63 may include a display device such as a display screen.

[0051] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for position alignment of a three-dimensional model of a customized hearing aid based on multi-feature fusion according to any embodiment.

[0052] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0053] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0054] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0055] Computer program code for performing the operation of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as C or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0056] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, ordinary technicians in this field should understand that they can still modify the technical solutions recorded in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the technical solutions of the embodiments of the present invention.

Claims

1. A method for aligning the position of a three-dimensional model of a customized hearing aid based on multi-feature fusion, characterized in that: include: Obtaining an operating model and a template model for customizing an ear sample for a hearing aid, wherein each model includes an ear canal portion, an auricle portion, and a model bottom, wherein the model bottom is used to support the ear canal and the auricle portion when the model is laid flat; Align the centroids of the two models with the coordinate origin of the three-dimensional space; Calculate the average curvature and local geometric structure information of each vertex in the two aligned models, and arrange the calculation results into local feature vectors of each vertex; The average curvature of each vertex is converted into RGB color, and the continuous area of ​​the same color with the largest surface area in each model after alignment is identified as the bottom of the model; the two aligned models are rotated once with the center of mass as the center, so that the bottom of each model is parallel to the xy plane of the three-dimensional space; the two models after the rotation are rotated twice around the z axis with the center of mass as the center, so that the projection point of the highest point of the two models on the xy plane is on the same straight line with the origin of the coordinate system; By minimizing the Chamfer distance, the two models after the secondary rotation are aligned with the ear mold posture, and the rotation angle of the operation model in each direction is constrained to be less than a set threshold during the alignment; For a specific position on the registered template model, a ray is emitted along the vertex normal at the specific position, and the projection point of the ray on the registered operation model is determined as the preliminary alignment vertex of the specific position; the similarity between the feature vectors of the preliminary alignment vertex and multiple neighboring vertices around it and the feature vector of the vertex at the specific position is calculated respectively, and the vertex with the highest similarity is identified as the final alignment vertex of the specific position.

2. The method according to claim 1, characterized in that The step of aligning the centroids of the two models with the coordinate origin of the three-dimensional space includes: Reconstruct the surface meshes of the two models respectively so that the difference in the number of meshes between the two models is less than the set threshold; The two reconstructed models are moved separately so that the centroids of the two models are aligned with the coordinate origin of the three-dimensional space.

3. The method according to claim 2, characterized in that Before the two reconstructed models are moved respectively to align the centroids of the two models with the coordinate origin of the three-dimensional space, the method further includes: The centroid of any model is calculated according to the following formula: in, is the total number of vertices of any model, is the three-dimensional coordinate of the i-th vertex, is the three-dimensional coordinate of the center of mass.

4. The method according to claim 1, characterized in that: The calculation of the average curvature and local geometric structure information of each vertex in the two aligned models, and arranging the calculation results into local feature vectors of each vertex, includes: For the two aligned models, calculate the average curvature and normalized normal vector of each vertex respectively; Using principal component analysis, the local covariance matrix and multiple eigenvalues ​​of each vertex are calculated; Use the FPFH algorithm to calculate the normal vector angle of each vertex and the relative distance of the neighboring points; The average curvature, normalized normal vector, eigenvalues, normal vector angle and relative distance of neighboring points of each vertex are arranged in sequence to form the local eigenvector of each vertex.

5. The method according to claim 4, characterized in that The step of calculating the average curvature of each vertex includes: According to the following formula, calculate the average curvature of each vertex: in, Indicates i The average curvature of the vertices, Indicates i The sum of the angles of the triangles around the vertex, It is i The neighborhood area around a vertex.

6. The method according to claim 1, characterized in that The step of rotating the two aligned models with the center of mass as the center so that the bottom of each model is parallel to the xy plane of the three-dimensional space includes: Select three points at random from the flat bottom contours of the two aligned models to form two planes respectively; The two aligned models are rotated once with the center of mass as the center, so that the two planes are parallel to the xy plane of the three-dimensional space and are located below the xy plane.

7. The method according to claim 1, characterized in that The method of performing ear mold posture registration on the two models after the secondary rotation by minimizing the Chamfer distance, wherein the rotation angle of the operation model in each direction is constrained to be less than a set threshold, comprises: Iteratively solve the rotation matrix centered on the center of mass , so that ,in, Represents the vertex set of the template model, A vertex set representing the operation model, Indicates that the model will be operated according to the rotation matrix T The rotated vertex set; represents the Chamfer distance between two point sets, , Represents the vertices of the template model, Indicates the number of vertices of the template model, express The vertices in express The number of vertices in ; After each iteration, check Relative to Whether the rotation angle around each coordinate axis exceeds the set threshold; if the rotation angle around any coordinate axis exceeds the set threshold, stop rotating along any coordinate axis in the next iteration .

8. The method according to claim 1, characterized in that The respectively calculating the similarity between the feature vectors of the preliminary aligned vertex and a plurality of neighboring vertices around it and the feature vector of the vertex at the specific position includes: Calculating the cosine similarity between the feature vector of the preliminary aligned vertex and the feature vector of the vertex at the specific position; The cosine similarities between the feature vectors of a plurality of neighborhood vertices around the preliminary aligned vertex and the feature vector of the vertex at the specific position are calculated respectively.

9. An electronic device, characterized in that: include: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method for position alignment of a customized hearing aid three-dimensional model based on multi-feature fusion as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the program is executed by a processor, the method for position alignment of a three-dimensional model of a customized hearing aid based on multi-feature fusion as described in any one of claims 1-8 is implemented.

Images