Grid-conditioned re-topology method and system for a three-dimensional model of a hearing aid

Abstract

The application discloses a grid condition re-topology method and system for a hearing aid three-dimensional model, and belongs to the technical field of 3D modeling. The method firstly reads point clouds obtained by performing ear canal scanning on the ear of a target user, generates a hearing aid ear mold model in the form of a triangular grid, and standardizes the model; then inputs the standardized hearing aid ear mold model into a multi-scale grid feature coding network, extracts sub-component level features, component level features and global features of an ear anatomical structure in turn in a way of expanding a receptive field layer by layer, and fuses the features into multi-scale fusion features; and then inputs a pre-specified target quadrilateral number as condition information into a condition quadrilateral grid generation model together with the multi-scale fusion features, generates a quadrilateral patch set through re-topology, and finally outputs the quadrilateral patch set. The application can realize end-to-end automatic re-topology of a hearing aid ear mold model, and greatly improves the efficiency and quality of hearing aid individual design.

Description

Technical Field

[0001] This invention relates to the field of 3D modeling technology, and specifically to a method and system for automatically converting a triangular mesh of a hearing aid model into a specified number of quadrilateral meshes. Background Technology

[0002] In computer graphics, meshes, as a core representation of 3D models, are mainly divided into triangular meshes and quadrilateral meshes based on the number of vertices in their facets. In the personalized modeling and structural design of hearing aid shells, quadrilateral meshes exhibit several significant advantages over triangular meshes. Because hearing aid products need to conform to the complex curved surfaces of the human ear while also considering the layout of internal components, the regular distribution of vertices and edges in quadrilateral meshes facilitates parametric adjustments and surface editing, thereby improving modeling accuracy and design efficiency. Their excellent adaptability in subdivision surface algorithms helps achieve more natural deformation and animation binding, meeting the needs of hearing aids in virtual try-on and dynamic simulation. Furthermore, in texture mapping and normal reconstruction, quadrilateral meshes can effectively control error propagation, enhancing the accuracy and stability of the model's geometric expression, which is particularly crucial for hearing aid appearance rendering and structural optimization.

[0003] Currently, a large amount of basic 3D data used for hearing aid modeling, such as point clouds obtained through ear canal scanning devices or ear templates in open-source model libraries, typically exists in the form of triangular meshes. To meet the high-precision geometric modeling requirements of customized hearing aid design, these triangular meshes need to be converted into quadrilateral meshes. Existing conversion technologies mainly fall into two categories: traditional non-learning methods and deep learning methods. Traditional methods often rely on local geometric heuristics, which are difficult to adapt to the multi-scale surface features of ear models, resulting in discontinuous generated mesh structures and an inability to flexibly control output complexity based on the number of target quadrilaterals. For example, the face merging strategy based on Delaunay triangulation requires frequent manual parameter tuning, limiting its application in automated hearing aid modeling processes. While deep learning methods can extract mesh features to assist in conversion, they still suffer from problems such as insufficient utilization of multi-scale information, uncontrollable output mesh quantity, poor mesh quality, and non-differentiable optimization. This leads to non-flat faces and topological anomalies in the conversion results, affecting the structural consistency of the hearing aid model and making it difficult to adapt to the accuracy requirements of different design stages.

[0004] In summary, existing triangle-to-quadrilateral mesh conversion technologies are still insufficient to simultaneously achieve controllable quantity, geometric accuracy, high quality, and efficient automation of quadrilateral mesh generation in hearing aid modeling scenarios. There is an urgent need to develop an innovative solution that integrates multi-scale feature extraction, conditional generation mechanisms, and differentiable optimization strategies to systematically overcome existing technical bottlenecks and promote the advancement of hearing aid digital modeling technology. Summary of the Invention

[0005] To address the issues of insufficient adaptability to complex geometries and poor output mesh quality in the process of retopologizing triangular meshes to quadrilateral meshes during hearing aid modeling, this invention proposes a conditional retopologizing method and system for hearing aid 3D models. This invention can efficiently and accurately convert triangular meshes obtained from ear scans into high-quality quadrilateral meshes based on the number of quadrilateral faces specified by the user. This supports refined editing, structural smoothing, and parametric design of the hearing aid shell, effectively improving the efficiency and geometric quality of personalized hearing aid modeling.

[0006] The specific technical solution adopted in this invention is as follows:

[0007] In a first aspect, the present invention provides a mesh-conditional retopology method for three-dimensional models of hearing aids, comprising:

[0008] S1. Read the point cloud obtained by scanning the ear canal of the target user's ear, generate a triangular mesh-style hearing aid ear mold model and standardize it;

[0009] S2. Input the standardized hearing aid earmold model into the multi-scale grid feature encoding network. First, map the vertex coordinates to the initial vertex features. Then, input it into the multi-scale graph convolutional network to extract the sub-part-level features, part-level features and global features of the ear anatomy in a way that expands the receptive field layer by layer. Finally, perform global average pooling on the features of each scale and then fuse them into multi-scale fused features.

[0010] S3. Input the pre-specified number of target quadrilaterals and the multi-scale fusion features into the conditional quadrilateral grid generation model. First, map the number of target quadrilaterals into a conditional vector and then fuse it with the multi-scale fusion features through affine transformation. Then, concatenate it with the index encoding features of each quadrilateral grid vertex and input it into the coordinate decoder to predict the vertex coordinate set of the quadrilateral grid. Then, take each quadrilateral grid vertex as the center vertex and calculate the mixed distance between the center vertex and its neighboring vertices, which integrates spatial proximity and normal consistency. Select the neighboring vertices with the smallest mixed distance and establish candidate edges with the center vertex. Calculate the existence probability of each candidate edge. Select a set of non-intersecting candidate quadrilaterals as a quadrilateral patch set from all quadrilaterals that can be formed by all candidate edges of all vertices. Maximize the sum of the existence probabilities of this set of candidate quadrilaterals and satisfy the pre-specified number of target quadrilaterals. The existence probability of each candidate quadrilateral is the product of the existence probabilities of its four edges.

[0011] S4. Combine the vertex coordinate set and quadrilateral face set of the quadrilateral mesh to output the hearing aid ear mold model in quadrilateral mesh form, which is used for personalized customization design of hearing aid shell.

[0012] As a preferred embodiment of the first aspect, the standardization method of the hearing aid earmold model is as follows: firstly, spatial registration is performed on the hearing aid earmold model in the form of a triangular mesh, the centroid of the model is moved to the origin of the coordinate system through a translation operation, and then scale normalization is performed to scale the spatially registered model to a unit sphere space according to a preset ratio.

[0013] As a preferred embodiment of the first aspect above, the multi-scale grid feature coding network includes a coordinate mapping unit, a multi-scale graph convolution unit, and a feature fusion unit;

[0014] In the coordinate mapping unit, the three-dimensional coordinates of each triangular mesh vertex of the hearing aid earmold model are input into a double hidden layer linear transformation network, thereby mapping them to the initial vertex features.

[0015] In the multi-scale graph convolutional unit, the hearing aid ear mold model containing vertex features is input into a three-layer graph convolutional network with progressively expanding receptive fields in the form of a graph structure. In each layer of the graph convolutional network, each triangular mesh vertex is used as the center vertex and the neighborhood point set is determined according to the receptive field radius of the current layer. Then, the node features of the neighborhood point set are aggregated to the center vertex. Thus, the first layer of the graph convolutional network extracts the sub-part-level features of the ear anatomy, the second layer of the graph convolutional network extracts the part-level features of the ear anatomy, and finally the third layer of the graph convolutional network extracts the global features of the ear.

[0016] In the feature fusion unit, the sub-component-level features, component-level features, and global features are first subjected to global average pooling, and then the importance coefficients of each scale feature are adaptively calculated by the attention mechanism. Then, the features are weighted and fused according to the importance coefficients. The fused feature vector is input into a multilayer perceptron for nonlinear transformation to obtain multi-scale fused features.

[0017] Furthermore, in the multi-scale graph convolutional unit, the receptive field radii of the three-layer graph convolutional network are 0.04~0.06, 0.09~0.11, and 0.19~0.21, respectively.

[0018] As a preferred embodiment of the first aspect above, the conditional quadrilateral mesh generation model includes a target quantity encoding unit, a vertex coordinate generation unit, and a topology connection prediction unit.

[0019] In the target quantity encoding unit, the pre-specified target quadrilateral quantity is input into the linear layer as a conditional control signal, thereby being encoded into a conditional vector.

[0020] In the vertex coordinate generation unit, the conditional vector and the multi-scale fusion feature are fused through affine transformation to obtain a conditional fusion vector. Then, the number of vertices N in the corresponding quadrilateral grid form of the hearing aid ear mold model is estimated according to the number of target quadrilaterals. The conditional fusion vector is concatenated with the index encoding feature of each quadrilateral grid vertex to obtain the corresponding vertex feature. Finally, the quadrilateral grid containing vertex features is constructed into a graph structure and input into a coordinate decoder composed of a graph convolutional network and a fully connected network. The vertex features of each quadrilateral grid vertex are mapped to three-dimensional coordinates to obtain the coordinate set of all quadrilateral grid vertices.

[0021] In the topology connectivity prediction unit, the mixing distance between each quadrilateral grid vertex and the other quadrilateral grid vertices is calculated sequentially, using each quadrilateral grid vertex as the center vertex. The mixing distance between any two quadrilateral grid vertices... and The mixed distance between the two vertices is obtained by multiplying the Euclidean distance between them by the normal inconsistency coefficient, where the normal inconsistency coefficient is calculated as follows: , Represents two quadrilateral grid vertices and The normal vectors of the nearest triangular mesh vertices on the triangular mesh-based hearing aid earmold model. and The dot product is calculated as follows: For each central vertex, according to the preset neighborhood point set size K, the K quadrilateral grid vertices with the smallest mixed distance to the central vertex are selected to establish candidate edges with the central vertex; then the coordinates of the two vertices connected by each candidate edge are mapped to vertex features and concatenated as edge features of the candidate edge, and then the edge features are input into the multilayer perceptron to obtain the existence probability of the candidate edge; the candidate edges of all quadrilateral grid vertices are added to the candidate edge set, and all simple rings of length 4 are searched as candidate quadrilaterals. The existence probability of each candidate quadrilateral is the product of the existence probabilities of the four edges. From all candidate quadrilaterals, a set of non-intersecting quadrilaterals is selected so that the existence probability of all quadrilaterals is maximized and the total number is equal to the pre-specified number of target quadrilaterals.

[0022] As a preferred embodiment of the first aspect above, the multi-scale grid feature encoding network and the conditional quadrilateral grid generation model need to be jointly trained on the training dataset before being used for actual inference, and the total loss function used for training is a weighted sum of vertex distance loss and normal consistency loss.

[0023] The vertex distance loss is the sum of the forward chamfer distance and the backward chamfer distance between the quadrilateral mesh generated by the model and the real quadrilateral mesh used as the truth label;

[0024] The normal consistency loss is the mean of the normal inconsistency deviation between each vertex in the real quadrilateral grid (which serves as the truth label) and its nearest neighbor vertex in the quadrilateral grid generated by the model. The normal inconsistency deviation is obtained by subtracting the dot product of the normal vectors of the two corresponding vertices in their respective quadrilateral grids from 1.

[0025] As a preferred embodiment of the first aspect above, the training dataset consists of ear mold data samples of different ages and genders, wherein each data sample contains a hearing aid ear mold model in the form of a triangular mesh, a hearing aid ear mold model in the form of a quadrilateral mesh obtained by a retopology tool, and the number of quadrilaterals contained therein.

[0026] In a second aspect, the present invention provides a computer electronic device, which includes a memory and a processor;

[0027] The memory is used to store computer programs;

[0028] The processor is configured to, when executing the computer program, implement the mesh-conditional retopology method for a three-dimensional model of a hearing aid as described in any of the first aspects above.

[0029] Thirdly, the present invention provides a personalized design system for hearing aid shells, comprising:

[0030] Ear canal scanning equipment is used to scan the ear of a target user and obtain point cloud data of the ear surface;

[0031] A data processing device is used to receive the ear surface point cloud data and a pre-specified number of target quadrilaterals, and output a hearing aid ear mold model in quadrilateral mesh form for the target user according to the mesh conditional retopology method for the three-dimensional model of the hearing aid as described in any of the above-mentioned first aspects; then, based on the hearing aid ear mold model in quadrilateral mesh form, the device performs shell removal, vent opening and digital assembly of internal electronic components according to the input operation instructions to generate a digital model of the hearing aid shell;

[0032] A 3D printing device is used to receive the digital model and form the hearing aid shell through 3D printing additive manufacturing.

[0033] As a preferred embodiment of the third aspect mentioned above, it also includes a preview interaction device for front-end display. While the data processing device outputs a hearing aid ear mold model in the form of a quadrilateral grid according to the pre-specified target number of quadrilaterals, it also generates a lightweight hearing aid ear mold model in the form of a quadrilateral grid by reducing the target number of quadrilaterals and sends it to the preview interaction device for real-time front-end display.

[0034] Compared with the prior art, the present invention has the following advantages:

[0035] This invention effectively captures multi-level geometric features, from local anatomical details of the ear to the overall ear contour and spatial relationships of components, through a multi-scale feature encoder. It demonstrates excellent feature extraction capabilities for complex curved structures such as the concha and ear canal. For the first time, this invention introduces the number of target quadrilaterals as an explicit conditional signal into the mesh generation process, allowing for precise control of the output mesh density according to hearing aid design requirements, adapting to the accuracy requirements of different design stages, from initial fitting to fine-tuning. Simultaneously, this invention employs a multi-task training mechanism that integrates geometric reconstruction loss and normal consistency constraints, ensuring that the generated quadrilateral mesh maintains a high degree of consistency with the ear's geometry while possessing a regular topological structure and smooth surface quality. This provides a high-quality mesh foundation for subsequent parametric editing, structural mechanics simulation, and 3D printing of the hearing aid shell. The entire process of this invention achieves end-to-end automated retopology, significantly reducing manual interaction and parameter adjustments in traditional methods, and greatly improving the efficiency and quality of personalized hearing aid modeling.

[0036] In summary, in practical applications, this invention can transform the unstructured triangular mesh of ear canal scanning into a high-quality quadrilateral mesh, thereby achieving comprehensive optimization in the three dimensions of hearing aid design, processing, and finished product performance. It has significant industrial application value in the entire process of digital manufacturing of hearing aid shells and customized ear molds. Attached Figure Description

[0037] Figure 1 A flowchart illustrating the steps of a mesh-conditional retopology method for 3D models of hearing aids;

[0038] Figure 2 This is a schematic diagram of the components of a computer electronic device;

[0039] Figure 3 A schematic diagram of the components of a system for personalized design of hearing aid shells;

[0040] Figure 4 The triangular mesh ear model serves as the basis for retopology in this embodiment;

[0041] Figure 5 The example shows a quadrilateral mesh ear model obtained by retopology using Blender's built-in retopology tool.

[0042] Figure 6 The quadrilateral mesh ear model is obtained by retopology using the method of the present invention in this embodiment. Detailed Implementation

[0043] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. Technical features in various embodiments of the present invention can be combined accordingly without mutual conflict.

[0044] This invention provides a mesh-conditional retopology method for 3D models of hearing aids. This method automatically converts the mesh model of the hearing aid earmold from a common triangular mesh to a quadrilateral mesh with a specified number of quadrilaterals according to the required resolution accuracy, providing a reliable and stable model foundation for the digital manufacturing process of the hearing aid shell and customized earmold. The specific implementation of this mesh-conditional retopology method is described in detail below. It should be noted that the triangular mesh and quadrilateral mesh in this invention are essentially different mesh types of hearing aid earmolds; therefore, in some descriptions of this invention, they are also referred to as triangular mesh model and quadrilateral mesh model, respectively, and their meanings are equivalent.

[0045] As a preferred embodiment of the present invention, the present invention provides a mesh-conditional retopology method for a three-dimensional model of a hearing aid, which includes the following steps S1 to S4, as follows: Figure 1 As shown in the diagram. The specific implementation of each step is explained below.

[0046] S1. Read the point cloud obtained by scanning the ear canal of the target user's ear, generate a triangular mesh-style hearing aid ear mold model, and standardize it.

[0047] It should be noted that the target users referred to in this invention are those who require personalized hearing aid earmolds. Ear canal scanning can be performed using existing ear canal scanning equipment. The specific equipment and ear canal scanning method generally include the external auditory canal, concha, cymba concha, auricle, tragus, antihelix, and other anatomical structures of the ear. The specific equipment and scanning method can be determined and operated by a professional fitting specialist based on the actual type of hearing aid being customized.

[0048] It should be noted that the point cloud obtained from the ear canal scan can be used to reconstruct a triangular mesh-based hearing aid earmold model using specialized software tools (such as ShellDesigner, EarMouldDesigner, Rapid Shell Modeling, etc.) or mesh generation algorithms (such as the Delaunay triangulation algorithm). This generation process is existing technology and will not be described in detail here. The triangular mesh-based hearing aid earmold model can be represented as follows: ,in Represents a vertex set. Represents an edge set. This represents a set of triangular facets.

[0049] Furthermore, the aforementioned standardization method for hearing aid earmold models needs to be designed reasonably based on the actual obtained mesh data. As a preferred embodiment of the present invention, the standardization method for hearing aid earmold models is as follows: First, spatial registration is performed on the triangular mesh-type hearing aid earmold model by moving the model's centroid to the origin of the coordinate system through a translation operation, thereby achieving spatial alignment; then, scale normalization is performed, scaling the spatially registered model to a unit sphere space according to a preset ratio to unify the coordinate reference system and scale benchmark.

[0050] S2. Input the standardized hearing aid ear mold model into the multi-scale grid feature encoding network. First, map the vertex coordinates to the initial vertex features. Then, input it into the multi-scale graph convolutional network to extract the sub-part-level features, part-level features and global features of the ear anatomy in a way that expands the receptive field layer by layer. Finally, perform global average pooling on the features of each scale and then fuse them into multi-scale fused features.

[0051] It should be noted that the multi-scale features mentioned in this invention are divided into three progressive scales: sub-component level, component level, and global level. Here, "component" refers to the anatomical structures of the ear, such as the external auditory canal, concha, cymba conchae, auricle, tragus, and antihelix. Correspondingly, sub-component level features refer to the local detailed features of the ear's anatomical structures, such as the edge of the auricle, the curvature of the ear canal, and the subtle convexity and concavity of the concha, which are low-level features; component level features refer to the morphological features of the anatomical structures and component-level geometric information such as local bulges and depressions, which are mid-level features; global features refer to the spatial relationships between the entire ear outline and its components, which are high-level features.

[0052] As a preferred embodiment of the present invention, the aforementioned multi-scale grid feature coding network includes a coordinate mapping unit, a multi-scale graph convolution unit, and a feature fusion unit. The multi-scale fusion feature extraction process is realized through the network modules built into the three units. The specific internal calculation process of the three units is described below.

[0053] In the aforementioned coordinate mapping unit, the three-dimensional coordinates of each triangular mesh vertex of the hearing aid earmold model are input into a double-hidden-layer linear transformation network (i.e., a fully connected network consisting of two fully connected layers), thereby mapping them to the initial vertex features.

[0054] In the aforementioned multi-scale graph convolutional unit, the hearing aid ear mold model containing vertex features is input into a three-layer graph convolutional network with progressively expanding receptive fields in the form of a graph structure. In each layer of the graph convolutional network, each triangular mesh vertex is used as the center vertex, and the neighborhood point set is determined according to the receptive field radius of the current layer. Then, the node features of the neighborhood point set are aggregated onto the center vertex. Thus, the first layer of the graph convolutional network extracts the sub-part-level features of the ear anatomy, the second layer of the graph convolutional network extracts the part-level features of the ear anatomy, and finally, the third layer of the graph convolutional network extracts the global features of the ear.

[0055] In the aforementioned feature fusion unit, the sub-component-level features, component-level features, and global features are first subjected to global average pooling. Then, all vertex features at each scale are pooled into a feature vector. The importance coefficients of the features at each scale are then adaptively calculated by the attention mechanism. Finally, the feature vectors of the three scales are weighted and fused according to the importance coefficients. The fused feature vectors are then input into a multilayer perceptron (MLP) for nonlinear transformation to obtain multi-scale fused features.

[0056] It should be noted that the specific graph convolutional network form of the aforementioned multi-scale graph convolutional units can be optimized and selected according to actual conditions. Furthermore, to ensure accurate extraction of effective features at the three scales, the receptive field radius of the three-layer graph convolutional network also needs optimization. In the embodiments of this invention, a three-layer graph convolutional network based on GraphSAGE can be used. When the grid model is scaled to a unit sphere space, the receptive field radii of the first, second, and third layers of the graph convolutional network are 0.04~0.06, 0.09~0.11, and 0.19~0.21, respectively, and more preferably 0.05, 0.1, and 0.2. Based on this set of receptive field radii, features at the desired scale can be extracted effectively.

[0057] S3. Input the pre-specified number of target quadrilaterals and the multi-scale fusion features into the conditional quadrilateral grid generation model. First, map the number of target quadrilaterals into a conditional vector and then fuse it with the multi-scale fusion features through affine transformation. Then, concatenate it with the index encoding features of each quadrilateral grid vertex and input it into the coordinate decoder to predict the vertex coordinate set of the quadrilateral grid. Then, take each quadrilateral grid vertex as the center vertex and calculate the mixed distance between the center vertex and its neighboring vertices, which integrates spatial proximity and normal consistency. Select the neighboring vertices with the smallest mixed distance and establish candidate edges with the center vertex. Calculate the existence probability of each candidate edge. Select a set of non-intersecting candidate quadrilaterals as a quadrilateral patch set from all quadrilaterals that can be formed by all candidate edges of all vertices. Maximize the sum of the existence probabilities of this set of candidate quadrilaterals and satisfy the pre-specified number of target quadrilaterals. The existence probability of each candidate quadrilateral is the product of the existence probabilities of its four edges.

[0058] It should be noted that the aforementioned pre-specified number of target quadrilaterals refers to the number of quadrilateral faces contained in the final retopology-generated quadrilateral mesh, as specified by the designer according to design requirements. For ease of description, this pre-specified number of target quadrilaterals will be denoted as... The number of vertices contained therein For quadrilateral meshes, the number of vertices... It can be based on the number of target quadrilaterals. The estimation can be expressed by the following formula: ,in This indicates rounding up to the nearest integer.

[0059] It should be noted that the coordinate decoder described above can be implemented using a neural network capable of feature mapping, such as a fully connected network consisting of multiple fully connected layers. Since the number of vertices in the quadrilateral grid has been estimated beforehand, these vertices can be predicted separately based on multi-scale fused features. The three-dimensional coordinates of the vertices of a quadrilateral grid. There are multiple estimation formulas, and the form given above is only one preferred method.

[0060] It should be noted that in the above operation of establishing candidate edges, each quadrilateral grid vertex needs to be selected based on the mixed distance to filter the neighboring vertices that need to establish candidate edges with itself. In the specific implementation, for all For each quadrilateral grid vertex, the mixing distance between each pair of vertices needs to be calculated. Then, for any quadrilateral grid vertex, it can be considered the center vertex. The remaining quadrilateral grid vertices are then sorted in ascending order of their mixing distance from the center vertex. The top K vertices with the smallest mixing distances are used to establish candidate edges between them and the center vertex, where K represents the size of the neighborhood set considered for each candidate edge, i.e., the number of vertices used to establish a candidate edge in a single step. Therefore, each quadrilateral grid vertex will have a series of candidate edges, and all candidate edges from all quadrilateral grid vertices can be added to the candidate edge set. From this set, all possible quadrilaterals can be filtered. Filtering quadrilaterals based on candidate edges can be implemented by finding simple cycles of length 4, where each simple cycle of length 4 is considered a possible quadrilateral. The product of the probabilities of the existence of the four sides of this quadrilateral is the probability of the quadrilateral existing.

[0061] In a preferred embodiment of the present invention, the conditional quadrilateral mesh generation model includes a target quantity encoding unit, a vertex coordinate generation unit, and a topology connectivity prediction unit. The generation process of the aforementioned quadrilateral patch set is achieved through the network modules built into these three units. The specific internal calculation process of the three units is described below.

[0062] In the aforementioned target quantity encoding unit, the pre-specified target quadrilateral quantity is input into the linear layer as a conditional control signal, thereby being encoded into a conditional vector.

[0063] In the vertex coordinate generation unit described above, the conditional vector and the multi-scale fusion feature are fused through affine transformation to obtain a conditional fusion vector. Then, the number of vertices N contained in the corresponding quadrilateral mesh hearing aid earmold model is estimated based on the number of target quadrilaterals, thereby constructing N initial quadrilateral mesh vertices. Each quadrilateral mesh vertex is assigned a unique index number in sequence. The conditional fusion vector and the index encoding feature of each quadrilateral mesh vertex are concatenated to obtain the vertex feature of each quadrilateral mesh vertex. Each quadrilateral mesh vertex is used as a graph node, and the quadrilateral mesh with vertex features is constructed as a graph structure and input into the coordinate decoder, which is composed of a graph convolutional network and a fully connected network. Neighbor information is aggregated through layer-by-layer graph convolutional layers, and vertex features are updated. Finally, the fully connected network maps the vertex features of each quadrilateral mesh vertex, which are aggregated through graph convolution, to three-dimensional coordinates, thereby obtaining the coordinate set of all quadrilateral mesh vertices.

[0064] It should be noted that the index encoding feature of the quadrilateral grid vertices, namely the position embedding of the index number, can be obtained by position encoding the index number. The position encoding can refer to the sine and cosine encoding method in Transformer.

[0065] In the aforementioned topology connectivity prediction unit, the mixing distance between each quadrilateral grid vertex and the other quadrilateral grid vertices is calculated sequentially, with each quadrilateral grid vertex as the center vertex. For any two quadrilateral grid vertices... and In this context, the mixed distance between two vertices is obtained by multiplying the Euclidean distance between the two vertices by the normal inconsistency coefficient, where the normal inconsistency coefficient is calculated as follows: , express and dot product, For the vertices of the quadrilateral grid The normal vector of the nearest triangular mesh vertex on a triangular mesh-based hearing aid earmold model. For the vertices of the quadrilateral grid The normal vector of the nearest neighboring triangular mesh vertex on the triangular mesh earmold model of the hearing aid. During the process of traversing all quadrilateral mesh vertices as center vertices, for each center vertex, according to a preset neighborhood point set size K, the K quadrilateral mesh vertices with the smallest mixed distance to the center vertex are selected to establish candidate edges with the center vertex. Then, the coordinates of the two vertices connected to each candidate edge are mapped to vertex features using a multilayer perceptron (MLP). These two vertex features are then concatenated as the edge features of the candidate edge, and the edge features of the candidate edge are input into the MLP to obtain the existence probability of the candidate edge. Finally, the candidate edges of all quadrilateral mesh vertices are added to the candidate edge set, and all simple cycles of length 4 are searched as candidate quadrilaterals. The existence probability of each candidate quadrilateral is the product of the existence probabilities of its four edges. From all candidate quadrilaterals, a set of disjoint quadrilaterals is selected such that the existence probability of all quadrilaterals is maximized and the total number of quadrilaterals in the set equals the pre-specified number of target quadrilaterals. Ultimately, the preset Unused vertices are automatically removed from the grid, resulting in a pure quadrilateral mesh with the number of vertices automatically satisfying the topological constraints.

[0066] S4. Combine the vertex coordinate set and quadrilateral face set of the quadrilateral mesh to output the hearing aid ear mold model in quadrilateral mesh form, which is used for personalized customization design of hearing aid shell.

[0067] It should be noted that the aforementioned multi-scale grid feature encoding network and conditional quadrilateral grid generation model need to be jointly trained on the training dataset before being used for actual inference. Only after training to achieve the required generation accuracy can they be formally deployed in practical applications. In this invention, the basic model training process is consistent with conventional methods, employing supervised training and backpropagation through a defined loss function to update the learnable parameters in the model using gradient updates.

[0068] In embodiments of the present invention, the specific training objective is to ensure that the generated mesh possesses correct local surface properties while maintaining the ear geometry; therefore, the total loss function used for training is... Vertex distance loss and normal consistency loss The weighted sum. Where, vertex distance loss... The sum of the forward chamfer distance and the backward chamfer distance between the quadrilateral mesh generated for the model and the real quadrilateral mesh used as truth labels. The normal consistency loss... Let be the mean of the normal inconsistency deviation between each vertex in the real quadrilateral mesh (which serves as the truth label) and its nearest neighbor vertex in the model-generated quadrilateral mesh, where the normal inconsistency deviation is calculated as follows: , Represents the dot product operation. and These represent the vertices in a real quadrilateral mesh. Its nearest neighbor vertex in the quadrilateral mesh generated by the model Their respective normal vectors. In the total loss function, Used for measurement and The difference in geometrical position is defined as the sum of the forward and backward chamfer distances. The forward chamfer distance ensures that each generated vertex has a corresponding position in the target mesh, while the backward chamfer distance ensures that all geometry of the target mesh is covered by the generated vertices, thus achieving geometrical alignment without explicit pairing. Used to measure the consistency between the generated mesh and the target mesh in the local surface normal direction.

[0069] Furthermore, in embodiments of the present invention, the training dataset can consist of earmold data samples of different ages and genders, making its sample distribution as generalizable as possible. Each data sample contains a hearing aid earmold model in the form of a triangular mesh. The hearing aid ear mold model obtained by the retopology tool is in the form of a quadrilateral mesh. as well as The number of quadrilaterals contained, therefore the final sample is a triple. The retopology tool used to convert triangular meshes into quadrilateral meshes can employ industry-recognized, mature software tools, such as the flow-based automatic retopology tool Quadriflow, to ensure the accuracy of truth labels.

[0070] It should be noted that the method steps shown in S1 to S4 above can essentially be implemented in the form of computer programs or software functional modules.

[0071] Therefore, based on the same inventive concept, this invention also provides a computer electronic device corresponding to the mesh conditional retopology method for a three-dimensional model of a hearing aid provided in the above embodiments, such as... Figure 2 As shown, it includes a memory and a processor;

[0072] The memory is used to store computer programs;

[0073] The processor is configured to implement the mesh-conditional retopology method for hearing aid 3D models as described above when executing the computer program.

[0074] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the technical solution of the present invention can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0075] Therefore, based on the same inventive concept, this invention provides a computer-readable storage medium corresponding to the mesh conditional retopology method for a three-dimensional model of a hearing aid. The storage medium stores a computer program, which, when executed by a processor, can realize the mesh conditional retopology method for a three-dimensional model of a hearing aid as described above.

[0076] Therefore, based on the same inventive concept, the present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, can implement the mesh conditional retopology method for a hearing aid 3D model as described above.

[0077] Specifically, in the computer-readable storage medium of the above three embodiments, the stored computer program is executed by a processor, which can perform the aforementioned steps S1 to S4.

[0078] It is understood that the aforementioned storage media may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. The storage media may also be various media capable of storing program code, such as USB flash drives, external hard drives, magnetic disks, or optical discs.

[0079] It is understood that the processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0080] In another preferred embodiment of the present invention, a personalized design system for hearing aid shells is also provided, comprising:

[0081] Ear canal scanning equipment is used to scan the ear of a target user and obtain point cloud data of the ear surface;

[0082] The data processing device is used to receive the point cloud data of the ear surface and the number of target quadrilaterals specified in advance, and output the hearing aid ear mold model in quadrilateral mesh form for the target user according to the mesh condition retopology method for the three-dimensional model of the hearing aid as described above; then, based on the hearing aid ear mold model in quadrilateral mesh form, the device performs shell removal, vent opening and digital assembly of internal electronic components according to the input operation instructions to generate a digital model of the hearing aid shell;

[0083] A 3D printing device is used to receive the digital model and form the hearing aid shell through 3D printing additive manufacturing.

[0084] It is important to note that the aforementioned ear canal scanning device, data processing device, and 3D printing device need to establish a communication connection via a data bus to enable the mutual transmission of data and signals. Both the ear canal scanning device and the 3D printing device can be commercially available equipment from the hearing aid fitting industry. The data processing device, however, can essentially be any device with data processing capabilities, such as a PC or server. The data processing device can provide data processing services as a local device or as a cloud server.

[0085] Furthermore, when the data processing device uses a server with high-performance data processing capabilities, the personalized design system for the hearing aid shell can also fully utilize the characteristic of the retopology method—that the number of target quadrilaterals can be flexibly adjusted according to needs—to set up a preview interaction device for front-end display. For example... Figure 3 As shown, both the preview interactive device and the 3D printing device can establish a communication connection with the data processing device. The data processing device preferably uses a cloud server or remote server to provide high-performance data processing capabilities for different preview interactive devices and 3D printing devices. Therefore, designers can specify a target number of quadrilaterals based on the high-precision mesh density required for actual 3D printing. The data processing device outputs a hearing aid earmold model in quadrilateral mesh form according to the target number of quadrilaterals specified by the designer. Simultaneously, it can also generate a lightweight hearing aid earmold model in quadrilateral mesh form by reducing the target number of quadrilaterals and calling the aforementioned mesh conditional retopology method for hearing aid 3D models. This lightweight quadrilateral mesh model is then sent to the preview interactive device for real-time front-end display.

[0086] Therefore, leveraging the adjustable number of quadrilaterals and the insensitivity of final output quality to the number of quadrilaterals in the mesh conditional retopology method of this invention, the system can output a lightweight model in the front-end interactive stage, achieving low-latency data transmission to the cloud and smooth 3D preview on the web page; while in the back-end manufacturing stage, it can output a high-density mesh to match the process requirements of high-precision 3D printing equipment, achieving a seamless connection from lightweight interaction to high-fidelity manufacturing. Designers can specify the optimal mesh number according to the specific needs of acoustic simulation or mechanical analysis, avoiding wasted computing power while ensuring computational accuracy, and significantly improving the iterative efficiency of customized design.

[0087] The retopology method for converting triangular meshes to quadrilateral meshes described in S1-S4 of this invention has significant industrial application value in the entire digital manufacturing process of hearing aid shells and customized ear molds. In practical applications, this invention can transform unstructured triangular meshes from ear canal scans into high-quality quadrilateral meshes, thereby achieving comprehensive optimization in three dimensions: design, processing, and finished product performance. First, in the digital design stage, the generated quadrilateral mesh ear mold model has an excellent topological structure, which can effectively eliminate surface artifacts caused by mesh irregularities, making the subdivided ear mold surface smoother. At the same time, the regular topological flow greatly improves the robustness of Boolean operations, ensuring that mesh damage or non-manifold errors are avoided when shelling the hearing aid shell, opening vents, and constructing internal suspension structures, significantly improving the efficiency of automated modeling. Second, in the additive manufacturing and post-processing stages, the printing slice paths obtained from the quadrilateral meshes generated based on retopology are more continuous and smooth, significantly reducing surface textures in the 3D printed product. This not only reduces the time and cost of subsequent manual polishing and grinding, but also effectively avoids uneven wall thickness or perforation caused by excessive grinding, thus significantly improving the yield rate of the production line while ensuring the precision of the shell. Finally, in terms of precision assembly and product performance, this technology ensures the accuracy of the internal cavity structure of the hearing aid, achieving zero-error assembly of precision components such as the microphone; at the same time, the smoother internal acoustic channel and optimized mechanical structure effectively improve the harmonic distortion problem of sound transmission and enhance the drop resistance of the shell, ultimately improving the patient's wearing comfort and hearing experience.

[0088] The present invention will further demonstrate, through a specific embodiment, the detailed implementation process and technical effects of the mesh-conditional retopology method and system for hearing aid 3D models shown in steps S1 to S4 above on a specific dataset, so as to facilitate understanding of the essence of the present invention.

[0089] Example

[0090] This invention discloses a mesh conditional retopology method for 3D models of hearing aids. Its core objective is to automatically convert the input triangular mesh of the ear into a high-quality quadrilateral mesh that conforms to the target number of facets. While maintaining the geometric accuracy of the ear's anatomical structure, it significantly improves the topological regularity and surface smoothness of the output mesh, providing support for the personalized design of hearing aid shells and subsequent engineering applications. The model forward inference process in this embodiment still follows steps S1-S4 as described above. However, to better understand the overall technical implementation process, this embodiment will detail the construction of the training dataset and the model training and inference process.

[0091] Step 1: Dataset construction and standardization.

[0092] The specific implementation steps of this step are as follows:

[0093] Step 1.1: Collected multiple 3D anatomical scanning databases of the ear, hearing aid model libraries, and related medical image reconstruction data, resulting in approximately 15,000 pairs of ear triangular mesh model samples, covering populations of different ages and genders. Based on the geometric morphology and structural complexity of the ear, these ear models can be divided into three categories: low-complexity models (face count ≤ 1,000, mainly the basic shape of the outer ear outline), medium-complexity models (1,000 < face count ≤ 10,000, such as those containing major structures like the concha and auricle), and high-complexity models (face count > 10,000, such as detailed models with complete ear canals and subtle anatomical features).

[0094] Step 1.2: Perform a geometric integrity check on the collected ear triangular mesh models. Use the Trimesh library to detect geometric anomalies such as non-manifold edges, self-intersecting surfaces, or holes in the models. Discard low-quality models with geometric anomalies and retain structurally complete and qualified triangular mesh models. These triangular mesh models can be regarded as hearing aid earmold models in the form of triangular meshes before retopology in each sample.

[0095] Step 1.3: In order to construct training samples with diverse topological densities, this embodiment uses each triangular mesh model... Number of pieces Dynamically determine the number of target quadrilaterals in the sample. The rules are as follows:

[0096]

[0097] in, This is the sampling scale factor, ranging from [0.1, 0.8], used to control the degree of simplification of the model after retopology. For example, when the original model contains 10,000 triangles, The corresponding target quadrilateral count is 2000; The corresponding target quantity is 5000.

[0098] When constructing the training data, 3 to 5 different samples are randomly sampled for each triangular mesh model. The values ​​are used to generate multiple sets of quadrilateral mesh models. To generate high-quality quadrilateral mesh models as ground truth labels for the samples, this embodiment uses the flow-field-based automatic retopology tool (Quadriflow) to retopologically analyze the triangular mesh models obtained in step 1.2. According to the preset target number of quadrilateral facets Convert to the corresponding quadrilateral mesh model The transformation process ensures the geometric consistency of the model and the integrity of key features. To facilitate the network's learning of geometric mapping relationships at different resolutions, the number of target quadrilateral facets for all samples is [number missing]. Perform min-max normalization:

[0099]

[0100] in and For all training samples in all datasets respectively The minimum and maximum values.

[0101] Step 1.4: Perform spatial registration and scale normalization on all ear model models in the dataset. First, calculate the centroid of each model's vertex. Then, move the model's center to the origin using a translation operation to achieve spatial alignment. Next, determine the scaling factor based on the maximum bounding box size of all model samples, and scale all models proportionally to a unit sphere space to unify the coordinate reference system and scale benchmark. The specific calculation process for spatial registration and scale normalization is as follows:

[0102]

[0103]

[0104] in: The vertex set in the triangular mesh ear model. denoted as the number of vertices in the earmold model, c is the centroid of the earmold model, and s is the scaling factor. For the original vertex The new vertex after translation operation, As vertices The new vertex coordinates after scaling. Note that each vertex participates in the calculation using three-dimensional coordinates.

[0105] Ultimately, each training sample used for training and testing the hearing aid model's mesh retopology scenario is a data triple. , among them The normalized value. This is the model after spatial registration and scale normalization. However, it's important to note that when inputting the mesh model into the subsequent neural network, it needs to be converted into a graph structure representation to adapt to the subsequent graph neural network processing. Each model... Represented as a graph structure In this system, vertices correspond to graph nodes, edges represent mesh adjacency relationships, and node features are initially represented as the 3D coordinates of the vertices. Specifically: For a set of nodes, each vertex of the mesh This corresponds to a node in a graph structure. Each node records the three-dimensional coordinates of the corresponding vertex. If the vertex and There is an edge in the triangular mesh of the original earmold model. Therefore, an undirected edge also needs to be established in the graph structure. .

[0106] Step 2: Multi-scale feature encoding.

[0107] A multi-scale grid feature encoding network adapted to the ear canal and hearing aid earmold structure is constructed. This network consists of the aforementioned coordinate mapping unit, multi-scale graph convolution unit, and feature fusion unit. The forward computation process within the three units is as follows:

[0108] Step 2.1: Input the preprocessed ear mesh vertex coordinates into a two-hidden-layer linear transformation network to map the three-dimensional coordinates into n-dimensional high-dimensional feature vectors as initial vertex features, providing basic feature input for subsequent multi-scale feature extraction of the ear mold anatomy. In this embodiment, each vertex of the ear mold model triangular mesh... The coordinates are represented as The vertex coordinates are mapped to a 128-dimensional feature space through two fully connected layers. The mapping transformation can be expressed as:

[0109]

[0110] in, As vertices The initial vertex feature representation, , and , The initial feature representation of all triangular mesh vertices as learnable parameters. The initial feature set that constitutes the entire triangular mesh model This serves as the input for subsequent graph convolutional networks.

[0111] Step 2.2: Transform the triangular mesh model initial feature set The first layer of the spatial relationship graph convolutional network is input to extract sub-part-level features of the ear's anatomical structure. This layer of graph convolution extracts low-level geometric features of the ear model by calculating the spatial relationships between vertices and their k-neighbor vertices, focusing on capturing local structural information such as the auricle edge, ear canal curvature, and subtle concavities and convexities of the concha. After the convolution output is processed by a nonlinear activation function and normalization, a low-scale representation of local geometric features, i.e., sub-part-level features, is obtained. .

[0112] Step 2.3: Sub-part level features The second layer of the spatial relationship graph convolutional network is then input to extract component-level features of the ear's anatomical structures. This layer, while maintaining the convolutional structure, expands the receptive range of the convolutional kernel to perceive a larger neighborhood region, recognizing the morphological features of key ear anatomical structures such as the tragus, antihelix, and cymba conchae, as well as component-level geometric information such as local bulges and depressions, thus obtaining component-level features. .

[0113] Step 2.4: Transfer component-level features The input continues into the third layer of the spatial relationship graph convolutional network to extract global features. This convolutional layer further expands the receptive field, fuses global topological information, and extracts global features containing the entire ear contour and the spatial relationships between various parts. This provides a complete structural context for the subsequent generation of quadrilateral meshes.

[0114] In this embodiment, the three-layer spatial graph convolutional network described above can be a graph convolutional network based on GraphSAGE, and the convolutional channel dimensions of the three layers are respectively... For the first The layer graph convolution operation, its convolution calculation formula is expressed as:

[0115]

[0116] in: Representing vertices The neighborhood set of points contained within the convolutional kernel centered on the convolutional kernel; the neighborhood set of points for the convolution needs to be based on the receptive field radius. Define: For each vertex It can satisfy Euclidean distance in the mesh model The set of vertices is used as the neighborhood point set. . and The first The weight matrix and bias of each convolutional layer. In this embodiment, the receptive field radius of the three convolutional layers... The receptive field radius of the graph convolutions is increased sequentially for the first, second, and third layers. They are respectively , , In addition, a feature normalization layer (Batch Normalization) can be added after each graph convolution operation to stabilize the feature distribution and improve the convergence speed of network training.

[0117] Step 2.5: Process the multi-scale feature sets obtained in steps 2.2 to 2.4. The unified integration process is as follows:

[0118] First, the features of all vertices at each scale. Global average pooling is performed to extract geometric semantic information at the global level. Specifically, let the... The vertex set of the mesh model at each scale is , , Each vertex The features are represented as Then the global semantic features corresponding to this scale Defined as the average of all vertex features:

[0119]

[0120] Secondly, using an attention mechanism, adaptive weights are learned based on features at each scale, and feature fusion is achieved through weighted summation, specifically as follows:

[0121]

[0122] in, For the output features of attention fusion, For learnable parameter vectors, For the first Global average pooling features at each scale The weight coefficients, and satisfying .

[0123] Finally, the fused output features A unified multi-scale fusion feature is obtained by performing a nonlinear transformation using a multilayer perceptron (MLP). Its dimensions The value is 128, and the nonlinear transformation formula is as follows:

[0124]

[0125] in, , and , These are learnable parameters.

[0126] This multi-scale fusion feature It comprehensively reflects the local geometric features and global topological features of the ear model, and can provide a semantically rich feature foundation for subsequent conditional mesh generation.

[0127] Step 3: Generate the conditional quadrilateral mesh.

[0128] A condition-sensitive quadrilateral mesh generation model is constructed, which consists of the aforementioned target quantity encoding unit, vertex coordinate generation unit, and topology connectivity prediction unit. This model can retopologically transform the original triangular mesh into a quadrilateral mesh model with a reasonable structure and satisfying quantity constraints. The forward computation process within the three units is as follows:

[0129] Step 3.1: Count the target quadrilaterals (i.e., the quadrilateral mesh model that needs to be retopologically generated) The number of quadrilateral faces contained therein, and it should be noted that during training and inference... The sources are different, during the training process Known from triplet samples During the reasoning process (This is an input specified by the designer) used as a conditional control signal; this parameter reflects the grid density and precision requirements needed for the hearing aid shell design. The target number of quadrilaterals... The scalar parameter is mapped to an n-dimensional condition vector using the linear transformation module. In this embodiment, the linear transformation module consists of two linear layers, with a ReLU activation function between the two layers, ultimately yielding... An embedding vector representing the target topological complexity.

[0130] Step 3.2: Fuse multi-scale features through affine transformation With condition vector Conditional feature fusion is performed, specifically as follows:

[0131] First, conditional sensing affine transformation is performed using a multilayer perceptron (MLP) to transform the data. Convert to Scaling parameters With offset parameter :

[0132]

[0133]

[0134] Recalculate the conditional fusion vector :

[0135]

[0136] in This indicates an element-wise multiplication operation. This fusion method enables conditional feature adaptive adjustment, allowing the conditional quadrilateral mesh generation model to dynamically control the output quadrilateral mesh density based on the number of target quadrilaterals.

[0137] Step 3.3: Based on conditional fusion vector Generate the vertex coordinates of the quadrilateral mesh.

[0138] First, based on the number of target quadrilaterals Estimate the number of target vertices that should be included in the quadrilateral mesh model generated by retopology. Therefore, the estimated number of target vertices is obtained. , corresponding to the construction of a containing The initial vertex coordinate set of the quadrilateral grid vertices .

[0139] Then, the conditional fusion vector Extending this to a local representation of each quadrilateral mesh vertex, i.e., the conditional fusion vector. copy Then, for each quadrilateral grid vertex, a position code corresponding to the vertex's own index number is concatenated to form the initial vertex feature matrix. :

[0140]

[0141] Where PE(i) represents the position code of vertex i in the quadrilateral mesh. For vertex i, its code is a... dimensional vector (in this embodiment) The feature values ​​of each dimension of the encoded vector can be obtained according to the sine and cosine coding method in Transformer, where the th... The formula for calculating dimension is:

[0142]

[0143]

[0144] in, The above operations ensure that each vertex shares global geometric context information in the initial stage. At the same time, positional encoding introduces the order difference between vertices, enabling the model to distinguish vertices in different positions and providing consistent feature input for subsequent coordinate decoding.

[0145] Finally, the quadrilateral mesh model In Each vertex is considered a graph node, and the vertex features are used as node features, thus transforming the quadrilateral mesh model. The graph is input into a coordinate decoder consisting of two graph convolutional layers and five fully connected layers. The information between vertices is first aggregated through the two graph convolutional layers to obtain the final vertex features of each vertex. Then, the vertex features of each vertex are processed through five fully connected layers. Mapped to the vertex in the quadrilateral mesh model 3D coordinates The set of vertex coordinates, composed of the three-dimensional coordinates of all the quadrilateral mesh vertices, conforms to the distribution of the ear-shaped surface. To accommodate model storage requirements, each vertex needs to be assigned a unique vertex index, which is recorded along with its coordinates. middle.

[0146] In this embodiment, the specific parameters of these five fully connected layers are shown in Table 1:

[0147] Table 1

[0148]

[0149] The final output is the 3D coordinates of each vertex:

[0150]

[0151] Furthermore, in order to maintain consistency with the coordinate space of the input mesh model, the range of the output 3D coordinates needs to be determined by... Function normalization to This ensures that all generated vertices are located inside the unit sphere space.

[0152] Step 3.4: Based on the vertex set generated in Step 3.3 Construct a quadrilateral mesh model The candidate edge set is determined and the patch connectivity of the mesh is predicted. The specific process is shown in steps 3.4.1 to 3.4.3:

[0153] Step 3.4.1: In this embodiment, a K-nearest neighbor graph structure based on normal constraints is used to construct the candidate edge set. Specifically, to avoid erroneous connections across surfaces at thin-walled structures such as the helix edge or concha wall, a hybrid distance metric that integrates spatial proximity and normal consistency is defined here. First, for the generated quadrilateral mesh model... any vertex in Use it as the center vertex and in the original triangular mesh Find the nearest point based on spatial distance of coordinates Then obtain In the triangular grid Reference normal vector on Subsequently, for each central vertex Based on quadrilateral mesh model The center vertex can be calculated by finding the reference normal vectors of all vertices in the center. and any other vertex Mixed distance between:

[0154]

[0155] in, This represents the Euclidean geometric distance between two vertices in a quadrilateral mesh model, which can be calculated from their respective three-dimensional coordinates. This is the normal penalty weight coefficient, used to control the degree of influence of normal difference on distance ranking; This is a penalty for inconsistency in legal direction. Represents the normal vector and normal vector dot product, and Vertices and In the original triangular mesh The normal vectors of their respective nearest neighboring triangular mesh vertices, and the range of values ​​for the dot product operation result are: Finally, based on the calculations... Sort all vertices in ascending order, select the K vertices with the smallest distance, and establish a connection between these K vertices and the current center vertex. The candidate edges are the central vertices. The corresponding candidate edge set can effectively eliminate interference points that are geometrically close but have discontinuous topological structures.

[0156] Based on the above process, for the quadrilateral mesh model any vertex in Each vertex can construct a candidate edge set with itself as the central vertex, used to record the possible edge connections between the central vertex and its surrounding neighboring vertices. Furthermore, for any pair of vertices... and The edge connecting these two vertices can be denoted as... , Corresponding edge features From vertex Vertex features and Vertex features It is pieced together, that is .

[0157] Step 3.4.2: Transform the quadrilateral mesh model The candidate edge sets corresponding to all vertices are merged. The edge features of each candidate edge in the merged candidate edge set are then input into a three-layer multilayer perceptron (MLP) to obtain a scalar score. The scalar score is then processed by the Sigmoid activation function to obtain the probability of the existence of the candidate edge. ,in, , indicating candidate edges The probability of existence in the final quadrilateral mesh model. All candidate edges in the merged candidate edge set. probability of existence Construct a set of edge probabilities .

[0158] Step 3.4.3: Obtain the edge probability set Then, it can be viewed as a graph, from which all simple cycles of length 4 are found and considered as candidate quadrilaterals. Specifically, by finding cycles of length 4 with no repeating vertices in the graph, each cycle representing a candidate quadrilateral, a set of candidate quadrilateral facets is generated. Assume the four vertices of the candidate quadrilateral are... , , , The probability of the existence of this candidate quadrilateral is the product of the probabilities of the existence of its four sides, which can be expressed as:

[0159]

[0160] Then, using integer programming or greedy optimization strategies, we can select from the set of candidate quadrilateral facets. Choose a set of disjoint quadrilateral faces from all candidate quadrilaterals within the quadrilateral. The goal is to maximize the probability of the existence of all quadrilaterals and ensure that the total number of quadrilaterals equals the pre-specified number of target quadrilaterals. In this embodiment, the preferred optimization strategy is a greedy optimization strategy, and its optimization objective can be expressed as:

[0161]

[0162] Based on quadrilateral patch set This can be used to further generate patch connectivity relationships. The face indices record the indices of the four vertices contained in each quadrilateral face.

[0163] Step 3.5: Combine the vertex coordinate sets obtained in Step 3.3 and Step 3.4 Connection relationship with surface Combine the elements to generate a complete quadrilateral mesh model output file, denoted as... The quadrilateral mesh model is derived from the original triangular mesh. Obtained through retopology. In subsequent neural network training, the generated quadrilateral mesh model... It can be compared with the quadrilateral mesh model that serves as the truth label in the triplet samples. They participate together in the calculation of the loss function and are updated through network parameters. as close as possible .

[0164] Step 4: Model training and differentiable optimization.

[0165] In this embodiment, to preserve the anatomical features of the earmold as much as possible and meet the accuracy requirements of hearing aid modeling (i.e., the number of target quadrilaterals specified by the user), the multi-scale grid feature encoding network and the conditional quadrilateral grid generation model need to be jointly trained on the training dataset before being used for actual inference. For this joint training task, this embodiment designs a method including vertex distance loss. and normal consistency loss The joint loss function is used to update the learnable network parameters during iterative optimization by calculating the joint loss and backpropagating gradients, thereby training a retopological neural network capable of adapting to ear models of varying complexity. The specific model joint training process is as follows:

[0166] Step 4.1: Divide the training dataset constructed in Step 1 into two parts: a training set and a validation set. The training set is used for learning and optimizing model parameters, while the validation set is used for evaluating model performance and verifying its generalization ability.

[0167] Step 4.2: For each training sample, the corresponding triplet The original ear mold triangular mesh Input a multi-scale grid feature encoding network to extract fused features At the same time, the number of target quadrilaterals As input conditional information, the quadrilateral mesh generation model is obtained, and finally, the quadrilateral mesh generated by the model is obtained. .

[0168] Step 4.3: Calculate vertex distance loss and normal consistency loss And a weighted joint loss function suitable for ear surface reconstruction is obtained. To evaluate the geometric and topological consistency between the generated mesh and the real mesh:

[0169]

[0170]

[0171]

[0172] in, Quadrilateral mesh generated for the model The set of vertices in the , For the true quadrilateral grid as truth labels The set of vertices in the array. , These are two weighting coefficients used to adjust the balance between the two losses, ensuring that the generated mesh maintains the ear geometry while possessing correct local surface properties. In this embodiment... , Take values ​​of 1.0 and 0.2 respectively.

[0173] Step 4.4: Jointly train the multi-scale grid feature encoding network and the conditional quadrilateral grid generation model using a gradient descent-based optimization algorithm. In each iteration, the training is performed according to the comprehensive loss function. The gradient results are used to update the parameters of the multi-scale feature encoder and conditional generator in the network, gradually reducing the difference between the generated grid and the target grid. When the decrease in validation set loss is lower than a set threshold or the maximum number of training epochs is reached, training is terminated, and the optimal weights are saved as the final model parameters to achieve high-precision, geometrically continuous retopology results. In this embodiment, the Adam optimizer is used, with an initial learning rate of 100%. Momentum parameters Batch Size is set to 8; the total number of training rounds is 200, with the learning rate decreasing by 0.5 times every 20 rounds; the upper limit of the gradient norm is 1.0 to prevent numerical oscillations; training is terminated early when the validation set loss decreases by less than 0.1% within 10 consecutive rounds.

[0174] Step 5: Practical application of the trained model

[0175] Once the model training in step 4 reaches the preset accuracy, the multi-scale grid feature encoding network and the conditional quadrilateral grid generation model can be combined into a retopological neural network for practical applications. In practical applications, the ear of the target user can be scanned using an ear canal scanning device to obtain point cloud data of the ear surface. Then, a triangular mesh hearing aid ear mold model is generated based on the point cloud data, and the same standardization operation (i.e., spatial registration and scale normalization) as the triplet samples in step 1 is performed. Then, the number of target quadrilaterals specified by the user is received, and the trained retopological neural network is called to retopologically regenerate the triangular mesh, generating a quadrilateral mesh hearing aid ear mold model. This ear mold model can be output and imported into hearing aid design software. According to the operation instructions input by the designer, the shell is removed, the vent is opened, and the internal electronic components are digitally assembled. After generating the digital model of the hearing aid shell, it is sent to a 3D printing device. The personalized hearing aid shell is printed through additive manufacturing process. Then, various electronic components and accessories are assembled onto the hearing aid shell to complete the initial processing of the hearing aid.

[0176] To better demonstrate the technical effect of the retopology method (referred to as the method of this invention for ease of description) shown in steps 1 to 5 of this embodiment, this embodiment compares the method of this invention with the retopology tool built into the 3D modeling software Blender on the same dataset. The server configuration for model deployment is as follows: four NVIDIA RTX 3090 graphics cards and an Intel Core i9 CPU. The batch size used for training is 8, and the training lasts for 500 epochs and 6 hours and 25 minutes. Both the method of this invention and the comparison method quantify the model effect using geometric error, mesh quality, and processing time. Geometric error uses Chamfer Distance to quantify the geometric shape difference between the retopological model and the original high-poly model. Mesh quality is measured by calculating the aspect ratio of the output mesh to determine whether there are many extremely thin and long meshes in the output mesh. Table 2 compares the processing results of the method of this invention and the built-in retopology tool in Blender as a comparison:

[0177] Table 2

[0178]

[0179] As shown in Table 2, the retopology results of this invention demonstrate that the method described in this paper outperforms the built-in retopology tool in Blender in both chamfer distance and mesh quality. The running time of this invention is largely unaffected by the number of meshes, while the built-in retopology tool in Blender is significantly affected by the number of meshes. In high-precision retopology scenarios (with more than 5000 target quadrilaterals), the processing time required by this invention is noticeably shorter than that of the built-in retopology tool in Blender.

[0180] In addition, to visually demonstrate the retopology effect of the method of this invention and the comparative method, Figure 4 Using the triangular mesh ear model shown as a base, two methods were applied to retopology at three target quadrilateral mesh counts of 1000, 5000, and 10000 to generate quadrilateral mesh models. Figure 5 This demonstrates the results of retopology using Blender's built-in retopology tool. Figure 6 The results of retopology based on the method of this invention are shown. It is clearly evident that, regardless of the number of target quadrilateral meshes, the method of this invention can ensure that the generated quadrilateral meshes essentially maintain the ear geometry and local surface characteristics that match the original triangular meshes—something that Blender's built-in retopology tools cannot achieve. Therefore, the method of this invention is particularly suitable for 3D hearing aid modeling scenarios where high controllability of mesh topology and geometric accuracy are required, and can provide technical support for the personalized design of high-precision 3D hearing aid models.

[0181] The embodiments described above are merely some preferred implementations of the present invention and are not intended to limit the invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, all technical solutions obtained through equivalent substitution or transformation fall within the protection scope of the present invention.

Claims

1. A mesh-conditional retopology method for three-dimensional models of hearing aids, characterized in that, include: S1. Read the point cloud obtained by scanning the ear canal of the target user's ear, generate a triangular mesh-style hearing aid ear mold model and standardize it; S2. Input the standardized hearing aid earmold model into the multi-scale grid feature encoding network. First, map the vertex coordinates to the initial vertex features. Then, input it into the multi-scale graph convolutional network to extract the sub-part-level features, part-level features and global features of the ear anatomy in a way that expands the receptive field layer by layer. Finally, perform global average pooling on the features of each scale and then fuse them into multi-scale fused features. S3. Input the pre-specified number of target quadrilaterals and the multi-scale fusion features into the conditional quadrilateral grid generation model. First, map the number of target quadrilaterals into a conditional vector and then fuse it with the multi-scale fusion features through affine transformation. Then, concatenate it with the index encoding features of each quadrilateral grid vertex and input it into the coordinate decoder to predict the vertex coordinate set of the quadrilateral grid. Then, take each quadrilateral grid vertex as the center vertex and calculate the mixed distance between the center vertex and its neighboring vertices, which integrates spatial proximity and normal consistency. Select the neighboring vertices with the smallest mixed distance and establish candidate edges with the center vertex. Calculate the existence probability of each candidate edge. Select a set of non-intersecting candidate quadrilaterals as a quadrilateral patch set from all quadrilaterals that can be formed by all candidate edges of all vertices. Maximize the sum of the existence probabilities of this set of candidate quadrilaterals and satisfy the pre-specified number of target quadrilaterals. The existence probability of each candidate quadrilateral is the product of the existence probabilities of its four edges. S4. Combine the vertex coordinate set and quadrilateral face set of the quadrilateral mesh to output the hearing aid ear mold model in quadrilateral mesh form, which is used for personalized customization design of hearing aid shell.

2. The mesh conditional retopology method for three-dimensional models of hearing aids as described in claim 1, characterized in that, The standardization method for the hearing aid earmold model is as follows: First, spatial registration is performed on the triangular mesh form hearing aid earmold model. The centroid of the model is moved to the origin of the coordinate system through translation operation. Then, scale normalization is performed to scale the spatially registered model to the unit sphere space according to a preset ratio.

3. The mesh conditional retopology method for hearing aid 3D models as described in claim 1, characterized in that, The multi-scale grid feature coding network includes a coordinate mapping unit, a multi-scale graph convolution unit, and a feature fusion unit; In the coordinate mapping unit, the three-dimensional coordinates of each triangular mesh vertex of the hearing aid earmold model are input into a double hidden layer linear transformation network, thereby mapping them to the initial vertex features. In the multi-scale graph convolutional unit, the hearing aid ear mold model containing vertex features is input into a three-layer graph convolutional network with progressively expanding receptive fields in the form of a graph structure. In each layer of the graph convolutional network, each triangular mesh vertex is used as the center vertex and the neighborhood point set is determined according to the receptive field radius of the current layer. Then, the node features of the neighborhood point set are aggregated to the center vertex. Thus, the first layer of the graph convolutional network extracts the sub-part-level features of the ear anatomy, the second layer of the graph convolutional network extracts the part-level features of the ear anatomy, and finally the third layer of the graph convolutional network extracts the global features of the ear. In the feature fusion unit, the sub-component-level features, component-level features, and global features are first subjected to global average pooling, and then the importance coefficients of each scale feature are adaptively calculated by the attention mechanism. Then, the features are weighted and fused according to the importance coefficients. The fused feature vector is input into a multilayer perceptron for nonlinear transformation to obtain multi-scale fused features.

4. The mesh conditional retopology method for hearing aid 3D models as described in claim 3, characterized in that, In the multi-scale graph convolutional unit, the receptive field radii of the three-layer graph convolutional network are 0.04~0.06, 0.09~0.11 and 0.19~0.21, respectively.

5. The mesh conditional retopology method for three-dimensional models of hearing aids as described in claim 1, characterized in that, The conditional quadrilateral mesh generation model includes an object quantity encoding unit, a vertex coordinate generation unit, and a topology connection prediction unit. In the target quantity encoding unit, the pre-specified target quadrilateral quantity is input into the linear layer as a conditional control signal, thereby being encoded into a conditional vector. In the vertex coordinate generation unit, the conditional vector and the multi-scale fusion feature are fused through affine transformation to obtain a conditional fusion vector. Then, the number of vertices contained in the corresponding quadrilateral grid form of the hearing aid ear mold model is estimated according to the number of target quadrilaterals. The conditional fusion vector is concatenated with the index encoding feature of each quadrilateral grid vertex to obtain the corresponding vertex feature. Finally, the quadrilateral grid containing vertex features is constructed into a graph structure and input into a coordinate decoder composed of a graph convolutional network and a fully connected network. The vertex features of each quadrilateral grid vertex are mapped to three-dimensional coordinates to obtain the coordinate set of all quadrilateral grid vertices. In the topology connectivity prediction unit, the mixing distance between each quadrilateral grid vertex and the other quadrilateral grid vertices is calculated sequentially, using each quadrilateral grid vertex as the center vertex. The mixing distance between any two quadrilateral grid vertices... and The mixed distance between the two vertices is obtained by multiplying the Euclidean distance between them by the normal inconsistency coefficient, where the normal inconsistency coefficient is calculated as follows: , Represents two quadrilateral grid vertices and The normal vectors of the nearest triangular mesh vertices on the triangular mesh-based hearing aid earmold model. and The dot product is calculated as follows: For each central vertex, according to the preset neighborhood point set size K, the K quadrilateral grid vertices with the smallest mixed distance to the central vertex are selected to establish candidate edges with the central vertex; then the coordinates of the two vertices connected by each candidate edge are mapped to vertex features and concatenated as edge features of the candidate edge, and then the edge features are input into the multilayer perceptron to obtain the existence probability of the candidate edge; the candidate edges of all quadrilateral grid vertices are added to the candidate edge set, and all simple rings of length 4 are searched as candidate quadrilaterals. The existence probability of each candidate quadrilateral is the product of the existence probabilities of the four edges. From all candidate quadrilaterals, a set of non-intersecting quadrilaterals is selected so that the existence probability of all quadrilaterals is maximized and the total number is equal to the pre-specified number of target quadrilaterals.

6. The mesh conditional retopology method for three-dimensional models of hearing aids as described in claim 1, characterized in that, Before being used for actual inference, the multi-scale grid feature encoding network and the conditional quadrilateral grid generation model need to be jointly trained on the training dataset in advance, and the total loss function used for training is a weighted sum of vertex distance loss and normal consistency loss. The vertex distance loss is the sum of the forward chamfer distance and the backward chamfer distance between the quadrilateral mesh generated by the model and the real quadrilateral mesh used as the truth label; The normal consistency loss is the mean of the normal inconsistency deviation between each vertex in the real quadrilateral grid (which serves as the truth label) and its nearest neighbor vertex in the quadrilateral grid generated by the model. The normal inconsistency deviation is obtained by subtracting the dot product of the normal vectors of the two corresponding vertices in their respective quadrilateral grids from 1.

7. The mesh conditional retopology method for three-dimensional models of hearing aids as described in claim 6, characterized in that, The training dataset consists of ear mold data samples of different ages and genders, where each data sample contains a hearing aid ear mold model in the form of a triangular mesh, a hearing aid ear mold model in the form of a quadrilateral mesh obtained by a retopology tool, and the number of quadrilaterals contained therein.

8. A computer electronic device, characterized in that, Including memory and processor; The memory is used to store computer programs; The processor is configured to, when executing the computer program, implement the mesh conditional retopology method for a three-dimensional model of a hearing aid as described in any one of claims 1 to 7.

9. A personalized design system for hearing aid shells, characterized in that, include: Ear canal scanning equipment is used to scan the ear of a target user and obtain point cloud data of the ear surface; A data processing device is used to receive the point cloud data of the ear surface and the number of target quadrilaterals specified in advance, and output the hearing aid ear mold model in quadrilateral mesh form for the target user according to the mesh conditional retopology method for hearing aid three-dimensional models as described in claim 1; then, based on the hearing aid ear mold model in quadrilateral mesh form, the device performs shell removal, vent opening and digital assembly of internal electronic components according to the input operation instructions to generate a digital model of the hearing aid shell; A 3D printing device is used to receive the digital model and form the hearing aid shell through 3D printing additive manufacturing.

10. The personalized design system for hearing aid shells as described in claim 9, characterized in that, It also includes a preview interaction device for front-end display. While the data processing device outputs a hearing aid ear mold model in the form of a quadrilateral grid according to the pre-specified target number of quadrilaterals, it generates a lightweight hearing aid ear mold model in the form of a quadrilateral grid by reducing the target number of quadrilaterals and sends it to the preview interaction device for real-time front-end display.

Images