Incremental confrontation reconstruction system and method for oral three-dimensional scanning data

Abstract

The invention discloses an incremental confrontation reconstruction system and method for oral three-dimensional scanning data, and relates to the technical field of computer graphics and medical image processing, and the system comprises an incremental scanning data acquisition module, a confrontation missing region complementation module, a geometric constraint surface reconstruction module, a multi-view feature fusion module and a topological optimization and intelligent matching module. A deep-coupled closed-loop collaborative architecture is formed among the modules, incremental data acquisition is realized through inter-frame dynamic registration, high-quality completion of a missing region is realized through a generative adversarial network, accurate surface reconstruction is realized through antagonistic geometric constraints, details of an occlusal surface are enhanced through multi-view attention fusion, and the accuracy of the occlusal surface is improved. Intelligent matching of treatment schemes is achieved through the tooth form knowledge base, the problems of motion artifacts, missing data and noise interference in oral cavity scanning are solved, and full-process intelligent processing from scanning to diagnosis is achieved.

Description

Technical Field

[0001] This invention relates to the fields of computer graphics and medical image processing technology, and in particular to an incremental adversarial reconstruction system and method for oral three-dimensional scan data. Background Technology

[0002] The rapid development of digital oral diagnostic and treatment technologies has driven the widespread application of intraoral scanners in clinical practice. Intraoral scanners use structured light or laser projection technology to acquire real-time three-dimensional morphological data of the oral cavity, providing a crucial data foundation for clinical applications such as orthodontic treatment planning, prosthesis fabrication, and implant guide planning. However, numerous technical challenges exist during oral scanning, including motion artifacts caused by the patient's tongue movements and swallowing actions, optical interference from saliva and blood, blind spots in narrow areas such as proximal tooth contact zones and gingival sulci, and noise introduced by scanner probe jitter. These issues severely affect the integrity and accuracy of the scan data.

[0003] In the prior art, Chinese invention patent CN117252847A discloses a method and device for detecting and identifying alveolar bone defects in maxillary anterior teeth. This method acquires CBCT images of maxillary anterior teeth in both sagittal and coronal sections, employs data augmentation strategies for image stitching and data amplification based on a self-learning data augmentation strategy, and then performs feature processing using an optimized YOLO v8 model. The final output indicates whether the alveolar bone is normal or has fenestrations or cracks. The backbone network of this method includes a cascaded structure of CSPModule and SPPF modules. The Neck network uses an AFPN progressive feature pyramid structure for feature fusion. The Head network employs decoupled classification and regression branches; the classification branch uses the VFL loss function, and the regression branch uses the DFL and CIoU loss functions.

[0004] The aforementioned existing technologies mainly suffer from the following technical problems: The solution targets the defect detection and recognition task of CBCT static images, employing a one-time processing mode, which is unsuitable for the incremental data processing needs of real-time acquisition scenarios using intraoral scanners; the solution focuses on feature extraction and classification of two-dimensional images, without addressing the issues of missing area completion and surface reconstruction in three-dimensional point cloud data; the solution lacks a dedicated processing mechanism for motion artifacts and noise in the scan data, making it difficult to cope with dynamic interference during oral scanning; and the solution fails to establish a link between the anatomical structure knowledge base and treatment plans, thus failing to achieve intelligent support for the entire process from scan data to diagnosis and treatment.

[0005] Therefore, there is an urgent need for a technical solution that can process incremental data in real time during oral scanning, effectively fill in missing areas, accurately reconstruct tooth surfaces, and intelligently match treatment plans. Summary of the Invention

[0006] The technical problem to be solved by this invention is to provide an incremental adversarial reconstruction system and method for oral three-dimensional scanning data, which can realize real-time incremental processing of oral scanning data, high-quality filling of missing areas, precise reconstruction of tooth surfaces, and intelligent matching and recommendation of treatment plans.

[0007] To address the aforementioned technical problems, the first aspect of this invention provides an incremental adversarial reconstruction system for three-dimensional oral cavity scanning data. This system includes an incremental scanning data acquisition module, an adversarial missing region completion module, a geometrically constrained surface reconstruction module, a multi-view feature fusion module, and a topology optimization and intelligent matching module.

[0008] The incremental scanning data acquisition module is used to receive the raw scanning point cloud output by the dental scanner, perform inter-frame dynamic registration processing on the raw scanning point cloud to obtain the registered point cloud sequence, and detect missing regions in the registered point cloud sequence based on point cloud density analysis to generate a missing region mask.

[0009] The adversarial missing region completion module receives the registered point cloud sequence and the missing region mask, generates point clouds for the missing regions through a generator network, and judges the authenticity of the generated point clouds through a discriminator network. Based on adversarial training, the parameters of the generator network are iteratively optimized, and the completed point cloud is output.

[0010] The geometrically constrained surface reconstruction module receives the completed point cloud, performs implicit surface fitting on the completed point cloud based on adversarial geometric constraint loss, generates a reconstructed surface mesh, and sends the geometric constraint feedback parameters to the adversarial missing region completion module to adjust the generation strategy of the generator network.

[0011] The multi-view feature fusion module receives the reconstructed surface mesh, performs multi-view projection on the reconstructed surface mesh to obtain a multi-view depth map, performs feature fusion on the multi-view depth map through an attention weighting mechanism, back-projects the fused features onto the reconstructed surface mesh, and outputs the fused enhanced mesh.

[0012] The topology optimization and intelligent matching module receives the fused enhanced mesh, performs mesh simplification and topology repair on the fused enhanced mesh, calculates the topology consistency index, sends the topology optimization feedback parameters to the geometric constraint surface reconstruction module to adjust the surface fitting parameters, outputs the final output model, and performs treatment plan matching on the final output model based on the tooth morphology knowledge base to generate a recommended list of treatment plans.

[0013] Preferably, in the incremental scanning data acquisition module, the registration convergence threshold for inter-frame dynamic registration processing is 0.01 mm to 0.1 mm, and the maximum number of iterations is 50 to 200. Preferably, in the point cloud density analysis, regions with local point cloud density lower than a preset density threshold are marked as missing regions, and the preset density threshold is 0.3 to 0.5 times the average point cloud density.

[0014] Preferably, in the adversarial missing region completion module, the generator network includes an encoder and a decoder, the encoder including 4 to 8 convolutional layers, and the decoder including 4 to 8 deconvolutional layers. Preferably, the discriminator network includes 5 to 10 convolutional layers. Preferably, the learning rate for adversarial training is set to 0.0001 to 0.001, and the number of training iterations is set to 1000 to 10000.

[0015] Preferably, in the geometrically constrained surface reconstruction module, the adversarial geometrical constraint loss includes a surface smoothness constraint term, a normal consistency constraint term, and a curvature continuity constraint term. Preferably, the implicit surface fitting uses a signed distance field representation, and the mesh resolution is set to 64 to 256.

[0016] Preferably, in the multi-view feature fusion module, the multi-view projection includes 6 to 12 evenly distributed view directions. Preferably, the attention weighting mechanism is a self-attention mechanism.

[0017] Preferably, in the topology optimization and intelligent matching module, mesh simplification reduces the number of mesh patches to 10% to 50% of the original number. Preferably, the tooth morphology knowledge base includes a standard tooth anatomy template library and a historical treatment protocol database.

[0018] The second aspect of the present invention provides an incremental adversarial reconstruction method for oral three-dimensional scanning data. The method includes an incremental scanning data acquisition step, an adversarial missing region completion step, a geometrically constrained surface reconstruction step, a multi-view feature fusion step, and a topology optimization and intelligent matching step. Each step corresponds to the function of the corresponding module in the above system.

[0019] The technical solution of this invention has the following beneficial effects:

[0020] The incremental scanning data acquisition module enables real-time dynamic registration of scan data and automatic detection of missing regions. Compared to the one-time batch processing mode of existing technologies, processing latency is reduced by more than 60%, supporting real-time feedback during the scanning process. The adversarial missing region completion module achieves high-quality point cloud completion based on generative adversarial networks. Compared to traditional interpolation methods, the geometric continuity between the completed region and the surrounding region is improved by more than 40%, effectively solving the problem of blind spots in scanning. The adversarial geometric constraint loss introduced by the geometric constraint surface reconstruction module improves the anatomical structure conformity of the reconstructed surface by more than 35%, effectively preserving the detailed features of the occlusal surface of the teeth. The attention-weighted fusion mechanism of the multi-view feature fusion module improves the accuracy of restoring fine structures such as occlusal surface grooves and fissures by more than 50%. The closed-loop feedback mechanism and knowledge base matching function established by the topology optimization and intelligent matching modules achieve intelligent processing throughout the entire process from scanning to diagnosis, with a treatment plan recommendation accuracy rate of over 85%. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the architecture of the incremental adversarial reconstruction system for oral three-dimensional scanning data provided in an embodiment of the present invention.

[0022] Figure 2 This is a flowchart illustrating the incremental adversarial reconstruction method for oral three-dimensional scanning data provided in this embodiment of the invention. Detailed Implementation

[0023] Please refer to the attached document. Figures 1-2 To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0024] Please see Figure 1 , Figure 1 This is a schematic diagram of the architecture of the incremental adversarial reconstruction system for three-dimensional oral cavity scan data provided in an embodiment of the present invention. Figure 1As shown, the system includes an incremental scanning data acquisition module 1, an adversarial missing region completion module 2, a geometrically constrained surface reconstruction module 3, a multi-view feature fusion module 4, and a topology optimization and intelligent matching module 5. These five modules form a deeply coupled closed-loop collaborative architecture. The output of the incremental scanning data acquisition module 1 serves as the input to the adversarial missing region completion module 2, the output of the adversarial missing region completion module 2 serves as the input to the geometrically constrained surface reconstruction module 3, the output of the geometrically constrained surface reconstruction module 3 serves as the input to the multi-view feature fusion module 4, and the output of the multi-view feature fusion module 4 serves as the input to the topology optimization and intelligent matching module 5. On the reverse path, the geometrically constrained surface reconstruction module 3 sends geometric constraint feedback parameters to the adversarial missing region completion module 2, and the topology optimization and intelligent matching module 5 sends topology optimization feedback parameters to the geometrically constrained surface reconstruction module 3, thus forming a dual closed-loop feedback structure.

[0025] The incremental scanning data acquisition module 1 is used to receive the original scanning point cloud output by the oral scanner, perform inter-frame dynamic registration processing on the original scanning point cloud to obtain the registered point cloud sequence, and detect the missing regions in the registered point cloud sequence based on point cloud density analysis to generate a missing region mask.

[0026] In the specific implementation, the dental scanner outputs raw scan point clouds at a rate of 15 to 60 frames per second. Each frame of raw scan point cloud contains 3,000 to 50,000 three-dimensional coordinate points, with a coordinate accuracy of 0.01 mm for each point. After receiving a new frame of raw scan point cloud, the incremental scanning data acquisition module 1 first performs preprocessing operations on that frame of point cloud. The preprocessing operations include statistical filtering to remove outliers and voxel downsampling to reduce the point cloud density. The number of neighboring points for statistical filtering is set to 20 to 50, and the standard deviation factor is set to 1.0 to 2.0. The voxel size for voxel downsampling is set to 0.1 mm to 0.5 mm.

[0027] After preprocessing, the incremental scanning data acquisition module 1 performs inter-frame dynamic registration processing on the current frame point cloud and the accumulated point cloud. The inter-frame dynamic registration processing adopts the incremental dynamic registration algorithm proposed in this invention. This algorithm first calculates the fast point feature histogram descriptor of the current frame point cloud, then searches for corresponding point pairs in the accumulated point cloud, and finally solves the registration transformation matrix through iterative optimization.

[0028] The process of solving the registration transformation matrix in the incremental dynamic registration algorithm is as follows. Let the point cloud of the current frame be... The accumulated point cloud is ,in and These represent the points in the current frame point cloud. The first point and the cumulative point cloud The three-dimensional coordinates of the points This indicates the number of points in the point cloud of the current frame. This represents the number of points in the cumulative point cloud. Registration transformation matrix. It is a 4×4 rigid transformation matrix, including the rotation matrix. Translation vector The registration objective function is defined as follows:

[0029] ,

[0030] in: This is the registration error function; For the first The weight coefficients of each pair of points are calculated based on the normal consistency and distance of the pair, and the values ​​range from 0 to 1. In the cumulative point cloud The corresponding nearest neighbor point; Denotes the Euclidean norm; The smoothing regularization coefficient has a value ranging from 0.01 to 0.1. This is a smoothing regularization term in the transformation matrix, used to constrain the continuity of transformations between adjacent frames.

[0031] Weighting coefficient The calculation formula is:

[0032] ,

[0033] in: For point Its corresponding point The Euclidean distance between them; The standard deviation parameter for distance weights ranges from 1 mm to 5 mm. For point The normal vector; For point The normal vector; This represents the non-negative value of the dot product of two normal vectors, used to measure the consistency of the normal vectors.

[0034] In a preferred embodiment of the invention, the registration convergence threshold is set to 0.01 mm to 0.1 mm. The iteration terminates when the change in registration error between two adjacent iterations is less than the registration convergence threshold. The maximum number of iterations is set to 50 to 200, and termination is forced when the maximum number of iterations is reached. Registration transformation matrix. Solve using the singular value decomposition method.

[0035] After completing the inter-frame dynamic registration process, the incremental scanning data acquisition module 1 merges the registered current frame point cloud into the accumulated point cloud to obtain an updated registered point cloud sequence. The registered point cloud sequence is stored in a time series format, with each frame retaining its timestamp and registration transformation matrix information.

[0036] In the missing region detection stage, the incremental scanning data acquisition module 1 detects missing regions in the registered point cloud sequence based on point cloud density analysis. The point cloud density analysis employs the adaptive missing region detection algorithm proposed in this invention. This algorithm first divides the registered point cloud sequence into a uniform three-dimensional voxel grid, then counts the number of points within each voxel as the local point cloud density, and finally marks voxels with densities lower than a preset density threshold as missing regions.

[0037] The specific implementation of the adaptive detection algorithm for missing regions is as follows. Let the spatial range of the registered point cloud sequence be... voxel size is Then the dimensions of the voxel grid are ,in , and The calculation method is similar. For a position in the voxel mesh... The voxels, their local point cloud density Defined as:

[0038] ,

[0039] in: voxels Point cloud density within the area, expressed in points per cubic millimeter; voxels The number of points contained within; The value is the side length of the voxel, ranging from 0.5 mm to 2 mm.

[0040] Average point cloud density The mean density of all non-empty voxels is calculated as follows:

[0041] ,

[0042] in: This represents the average point cloud density. The number of non-empty voxels; It is a set of indices for non-empty voxels.

[0043] In a preferred embodiment of the present invention, a preset density threshold is used. Set to 0.3 to 0.5 times the average point cloud density. When voxels... density Below the preset density threshold At that time, the voxel was marked as a missing region. Missing region mask. It is a binary array with the same dimensions as the voxel grid, where the element corresponding to the missing region has a value of 1 and the element corresponding to the non-missing region has a value of 0.

[0044] The incremental scanning data acquisition module 1 outputs the registered point cloud sequence and the missing region mask, which serve as the input data for the adversarial missing region completion module 2.

[0045] The adversarial missing region completion module 2 is used to receive the registered point cloud sequence and the missing region mask, generate point clouds for the missing regions through a generator network, and judge the authenticity of the generated point clouds through a discriminator network. Based on adversarial training, the parameters of the generator network are iteratively optimized, and the completed point cloud is output.

[0046] In its implementation, the adversarial missing region completion module 2 employs a conditional generative adversarial network architecture. The generator network takes the missing region mask and the neighboring point clouds in the registered point cloud sequence as conditional inputs to generate a completed point cloud that is geometrically continuous with the neighboring point clouds. The discriminator network receives the completed point cloud region and determines whether the region is real scanned data or generated data.

[0047] The generator network employs an encoder-decoder architecture. The encoder consists of six 3D convolutional layers, each followed by a batch normalization layer and a Leaky ReLU activation function. The encoder's input is the neighborhood point cloud surrounding the missing region, with the neighborhood range set to extend 3mm to 10mm beyond the missing region's boundary. The encoder encodes the neighborhood point cloud into a 256-dimensional latent feature vector. The decoder consists of six 3D deconvolutional layers, each followed by a batch normalization layer and a ReLU activation function. The decoder decodes the latent feature vector into a completed point cloud representing the missing region.

[0048] In a preferred embodiment of the present invention, the number of convolutional layers in the encoder is set to 4 to 8, and the number of deconvolutional layers in the decoder is set to 4 to 8. The kernel size of each layer of the encoder is 3×3×3, with a stride of 2 and padding of 1. The number of output channels in each layer of the encoder is 64, 128, 256, 512, 512, and 256, respectively. The kernel size of each layer of the decoder is 4×4×4, with a stride of 2 and padding of 1, respectively. The number of output channels in each layer of the decoder is 256, 128, 64, 32, 16, and 3, respectively. The three channels output by the last layer correspond to the three-dimensional coordinates of the completion points.

[0049] The discriminator network employs a multi-scale discrimination strategy, performing authenticity judgments at both the local neighborhood scale and the global dental arch scale. The local discriminator focuses on the geometric continuity between the completed region and the neighboring point cloud, while the global discriminator focuses on the morphological rationality of the overall point cloud after completion. The outputs of the two discriminators are weighted and summed to obtain the final discrimination result.

[0050] In a preferred embodiment of the invention, the discriminator network comprises 5 to 10 three-dimensional convolutional layers. The input range of the local discriminator is the missing region and its neighborhood, while the input range of the global discriminator is the entire dental arch region. The weight coefficients of the two discriminators are set to 0.6 and 0.4, respectively.

[0051] During adversarial training, the generator network and the discriminator network are optimized alternately. The generator network's loss function... Defined as:

[0052] ,

[0053] in: This represents the total loss of the generator; This is the adversarial loss of the generator, used to deceive the discriminator; To reconstruct the loss, the distance between the generated point cloud and the real point cloud is measured; The geometric consistency loss constrains the local geometric properties of the generated point cloud; The reconstruction loss weighting coefficient has a value range of 10 to 100; This is the weighting coefficient for geometric consistency loss, with a value ranging from 1 to 10.

[0054] Generator's adversarial loss Employing a least-squares generative adversarial network:

[0055] ,

[0056] in: To obtain from the standard normal distribution A sampled random noise vector; The conditional inputs include the mask for the missing region and the neighboring point cloud; For the generator based on noise and conditions The generated complete point cloud; The output of the discriminator has a value range of 0 to 1.

[0057] Reconstruction losses Use chamfer distance:

[0058] ,

[0059] in: To complete the generated point cloud; For realistic point clouds; and These represent the number of points in the two point clouds, respectively.

[0060] Loss function of discriminator network Defined as:

[0061] ,

[0062] in: To obtain from real data distribution The actual point cloud region sampled; the meanings of the other symbols are the same as before.

[0063] In a preferred embodiment of the invention, the learning rate for adversarial training is set to 0.0001 to 0.001, and the Adam optimizer is used, wherein... Set to 0.5. Set to 0.999. The number of training iterations is set to 1000 to 10000. In each iteration, the discriminator parameters are updated first, followed by the generator parameters.

[0064] The adversarial missing region completion module 2 also receives geometric constraint feedback parameters from the geometrically constrained surface reconstruction module 3. These parameters include the local curvature deviation value and normal deviation angle value of the completed region. When the geometric constraint feedback parameters indicate that the geometric characteristics of the completed result do not meet the requirements of the tooth anatomy, the adversarial missing region completion module 2 adjusts the generation strategy of the generator network, specifically by increasing the geometric consistency loss. Weighting coefficients To achieve this.

[0065] The adversarial missing region completion module 2 outputs the completed point cloud, which serves as the input data for the geometrically constrained surface reconstruction module 3. The completed point cloud is a complete point cloud obtained by merging the generated completed point cloud with the non-missing region point clouds in the original registered point cloud sequence.

[0066] The geometric constraint surface reconstruction module 3 receives the completed point cloud, performs implicit surface fitting on the completed point cloud based on adversarial geometric constraint loss, generates a reconstructed surface mesh, and sends the geometric constraint feedback parameters to the adversarial missing region completion module 2 to adjust the generation strategy of the generator network.

[0067] In its implementation, the geometrically constrained surface reconstruction module 3 employs a neural implicit surface representation method. This method represents the tooth surface as a signed distance field, learning a mapping function from three-dimensional coordinates to signed distance values ​​through a neural network. The isosurface of the signed distance field is the reconstructed tooth surface.

[0068] Implicit surface fitting employs a multilayer perceptron network as the distance field predictor. The multilayer perceptron network consists of 8 fully connected layers, with 512 neurons in each layer. The input is three-dimensional coordinates. The output is the signed distance value from this coordinate to the tooth surface. The network employs positional encoding to map low-dimensional inputs to a high-dimensional feature space, thereby enhancing the network's ability to represent high-frequency details.

[0069] The adversarial geometric constraint loss is calculated using an auxiliary discriminator network. This network takes a local surface patch of the reconstructed surface mesh as input and determines whether the patch conforms to the anatomical features of a tooth. The auxiliary discriminator network employs a PointNet architecture, enabling it to directly process point cloud data of irregular surface patches.

[0070] Adversarial geometric constraint loss Defined as:

[0071] ,

[0072] in: The total loss due to adversarial geometric constraints; For surface smoothness constraints; For normal consistency constraints; For curvature continuity constraints; This is the adversarial loss term for the auxiliary discriminator; , , , These are the weighting coefficients for each constraint term, which are set to 0.3, 0.3, 0.2, and 0.2 in a preferred embodiment of the present invention.

[0073] Surface smoothness constraint Gradient calculation based on signed distance field:

[0074] ,

[0075] in: For the neural network at sampling points The predicted signed distance value; The Hessian matrix at that point; This represents the number of sampling points; This represents the Frobenius norm.

[0076] Normal consistency constraint Measuring the consistency between the reconstructed surface normal and the input point cloud normal:

[0077] ,

[0078] in: This represents the number of surface sampling points; To reconstruct the surface at the first The normal vector at each sampling point is obtained by normalizing the gradient of the signed distance field; The input is the normal vector of the corresponding point in the point cloud.

[0079] Curvature continuity constraint term Constraining the curvature variation between adjacent surface patches:

[0080] ,

[0081] in: This represents the number of adjacent surface patch pairs. and The first Gaussian curvature values ​​at the boundary of adjacent surface patches.

[0082] In a preferred embodiment of the invention, the mesh resolution for implicit surface fitting is set to 64 to 256. The Marching Cubes algorithm is used to extract the isosurface mesh from the signed distance field. The isosurface is set to 0, indicating a surface with zero distance.

[0083] The geometrically constrained surface reconstruction module 3 also calculates geometrically constrained feedback parameters and sends them to the adversarial missing region completion module 2. The geometrically constrained feedback parameters include local curvature deviation values. and normal deviation angle value The local curvature deviation value measures the difference between the curvature of the filled area and the curvature of the standard tooth template. The normal deviation angle value measures the angular deviation between the normal of the filled area and the normal of the adjacent area. Exceeding the preset curvature deviation threshold or When the preset angle deviation threshold is exceeded, the geometric constraint surface reconstruction module 3 sends an adjustment signal to the adversarial missing region completion module 2.

[0084] The geometrically constrained surface reconstruction module 3 also receives topology optimization feedback parameters from the topology optimization and intelligent matching module 5. These parameters include the number of target mesh patches and the topology complexity constraint value. Based on these feedback parameters, the geometrically constrained surface reconstruction module 3 adjusts the resolution of the implicit surface fitting and the subdivision level of the Marching Cubes algorithm.

[0085] The geometrically constrained surface reconstruction module 3 outputs a reconstructed surface mesh, which serves as input data for the multi-view feature fusion module 4. The reconstructed surface mesh is in triangular patch mesh format and includes a list of vertex coordinates, a list of patch indices, and a list of vertex normals.

[0086] The multi-view feature fusion module 4 is used to receive the reconstructed surface mesh, perform multi-view projection on the reconstructed surface mesh to obtain a multi-view depth map, perform feature fusion on the multi-view depth map through an attention weighting mechanism, back-project the fused features onto the reconstructed surface mesh, and output the fused enhanced mesh.

[0087] In its implementation, the multi-view feature fusion module 4 first projects the reconstructed surface mesh from multiple perspectives. The projection perspectives are evenly distributed along the sphere, with the number of perspectives set to 6 to 12. Each perspective corresponds to a virtual camera, which is located on a sphere with a radius of 80 mm to 150 mm, centered on the dental arch. Each virtual camera faces the center of the dental arch, generating a depth map and a normal map.

[0088] In a preferred embodiment of the invention, the number of projection viewpoints is set to eight, located in eight uniformly distributed directions on the sphere. The resolution of the depth map and normal map is set to 256×256 pixels. The depth map records the distance value from the surface point corresponding to each pixel to the camera, and the normal map records the normal vector of the surface point corresponding to each pixel.

[0089] Feature extraction from multi-view depth maps employs a weighted convolutional neural network. The convolutional neural network consists of an encoder and a feature pyramid. The encoder uses a ResNet-34 architecture to encode the 256×256×1 depth map into feature maps of multiple scales. The feature pyramid integrates these feature maps at different scales, outputting a 64×64×256-dimensional feature tensor.

[0090] The attention weighting mechanism is a self-attention mechanism, which assigns fusion weights to each viewpoint by calculating the feature correlations between depth maps from different perspectives. The specific implementation of the multi-view attention fusion algorithm is as follows. Let the... The feature tensor of each perspective is ,in , , These represent the height, width, and number of channels of the feature map, respectively. First, the feature tensors for each viewpoint are flattened as follows: Then, the attention weight matrix is ​​calculated.

[0091] The formula for calculating attention weights is:

[0092] ,

[0093] in: From the perspective Perspective The attention weight matrix, with dimensions of ; and These are the query transformation matrix and the key transformation matrix, respectively, with dimensions of [dimensions missing]. ; The dimension of the key vector is set to 64. This represents an exponential function normalized by row.

[0094] Fusion feature vectors The calculation formula is:

[0095] ,

[0096] in: Total number of viewpoints; The value transformation matrix has dimension 1. ; The dimension of the value vector is set to 64.

[0097] The fused features are back-projected onto the reconstructed surface mesh using a differentiable rendering method. This method establishes a correspondence between pixels in the 2D feature map and points on the 3D surface, mapping the feature value of each pixel to its corresponding surface point. For cases where the same surface point is covered by projections from multiple viewpoints, a weighted averaging strategy is used to integrate features from different viewpoints. The weights are determined based on the angle between the viewpoint and the surface normal; the smaller the angle, the greater the weight.

[0098] The fusion-enhanced mesh adds a fusion feature vector to each vertex of the original reconstructed surface mesh. This fusion feature vector is used for subsequent detail enhancement processing. The detail enhancement processing employs a feature-guided mesh subdivision method, increasing the mesh density in regions where the fusion feature values ​​change drastically to more accurately represent fine structures such as grooves and cracks on the interlocking surface.

[0099] The multi-view feature fusion module 4 outputs a fused and enhanced mesh, which serves as input data for the topology optimization and intelligent matching module 5. The fused and enhanced mesh contains an enhanced list of vertex coordinates, a list of face indices, a list of vertex normals, and a list of vertex feature vectors.

[0100] The topology optimization and intelligent matching module 5 is used to receive the fused enhanced mesh, perform mesh simplification and topology repair on the fused enhanced mesh, calculate the topology consistency index, send the topology optimization feedback parameters to the geometric constraint surface reconstruction module 3 to adjust the surface fitting parameters, output the final output model, and perform treatment plan matching on the final output model based on the tooth morphology knowledge base to generate a treatment plan recommendation list.

[0101] In its implementation, the topology optimization and intelligent matching module 5 first performs mesh simplification on the fused and enhanced mesh. Mesh simplification employs a quadratic error metric method, reducing the number of mesh patches through iterative edge merging while maintaining the accuracy of the surface geometry.

[0102] The objective function for mesh simplification in the adaptive topology optimization algorithm is defined as:

[0103] ,

[0104] in: To merge edges into vertices Position error measurement; The quadratic error matrix is ​​obtained by adding the quadratic error matrices of the two vertices at the two ends of the edge.

[0105] In a preferred embodiment of the invention, mesh simplification reduces the number of mesh patches to 10% to 50% of the original number. The simplification ratio is determined based on the intended use of the final output model; for CAD / CAM fabrication of the restoration, which requires higher precision, the simplification ratio is set to 50%; for quick preview, which requires lower precision, the simplification ratio is set to 10%.

[0106] After mesh simplification, the topology optimization and intelligent matching module 5 performs topology repair on the simplified mesh. Topology repair includes three sub-steps: hole filling, non-manifold edge repair, and self-intersection detection and elimination. Hole filling uses a minimum area triangulation method to close the hole boundaries in the mesh. Non-manifold edge repair eliminates non-manifold edges by copying shared vertices. Self-intersection detection and elimination removes self-intersecting regions using Boolean operations.

[0107] Topological consistency metrics include Euler characteristic and genus number. Euler characteristic The calculation formula is:

[0108] ,

[0109] in: The number of vertices; Let the number of edges be denoted as '('). Let be the number of facets. For a closed mesh that is topologically equivalent to a sphere, the Euler characteristic should be 2.

[0110] genus number The calculation formula is:

[0111] ,

[0112] For a topologically correct tooth model, the genus number should be 0.

[0113] After calculating the topology consistency index, the topology optimization and intelligent matching module 5 generates topology optimization feedback parameters based on the index value and sends them to the geometrically constrained surface reconstruction module 3. The topology optimization feedback parameters include the number of target mesh patches. and topology complexity constraints When a topological anomaly is detected, the topology optimization feedback parameters instruct the geometrically constrained surface reconstruction module 3 to reduce the resolution of the implicit surface fitting or enhance the smoothness constraints to avoid generating topological defects.

[0114] After topology optimization is completed, the topology optimization and intelligent matching module 5 outputs the final output model. The final output model is a topologically correct and geometrically accurate editable 3D mesh model, which can be directly used for subsequent clinical applications such as orthodontic treatment design, prosthesis fabrication, or implant guide planning.

[0115] The treatment plan matching function is based on a tooth morphology knowledge base. This knowledge base consists of two parts: a standard tooth anatomy template library and a historical treatment plan database. The standard tooth anatomy template library stores standard anatomical templates for various tooth positions, including templates for central incisors, lateral incisors, canines, premolars, and molars. The historical treatment plan database stores clinically accumulated treatment cases, each including the patient's initial tooth morphology, the treatment plan used, and treatment outcome evaluation.

[0116] The specific implementation of the treatment plan similarity matching algorithm is as follows. First, the morphological similarity between the final output model and each template in the standard dental anatomy template library is calculated. The morphological similarity is calculated by a weighted combination of Hausdorff distance and chamfer distance.

[0117] Morphological similarity The calculation formula is:

[0118] ,

[0119] in: The value represents morphological similarity and ranges from 0 to 1. Hausdorff distance measures the maximum deviation between two grids. The chamfer distance measures the average deviation between two grids; and These are weighting coefficients, which are set to 0.1 and 0.5 respectively in the preferred embodiments of the present invention.

[0120] Hausdorf Distance The calculation formula is:

[0121] ,

[0122] in: and These are the vertex sets of two separate grids.

[0123] chamfer distance The calculation formula is:

[0124] ,

[0125] Based on the morphological similarity matching results, the topology optimization and intelligent matching module 5 retrieves historical cases with similar morphology from the historical treatment plan database and extracts their treatment plans as recommendation candidates. The treatment plan recommendation list is sorted in descending order of morphological similarity score, and the top 3 to 10 treatment plans with the highest similarity scores are output. Each recommended treatment plan includes information such as plan type, expected effect, reference case, and similarity score.

[0126] Please see Figure 2 , Figure 2 This is a flowchart illustrating the incremental adversarial reconstruction method for oral three-dimensional scan data provided in this embodiment of the invention. The method includes an incremental scan data acquisition step S1, an adversarial missing region completion step S2, a geometrically constrained surface reconstruction step S3, a multi-view feature fusion step S4, and a topology optimization and intelligent matching step S5. Each step corresponds to the function of the corresponding module in the system embodiment, forming a complete processing flow from scan data input to treatment plan output.

[0127] Step S1, incremental scanning data acquisition step: receive the original scanning point cloud output by the dental scanner, perform inter-frame dynamic registration processing on the original scanning point cloud to obtain the registered point cloud sequence, and detect the missing regions in the registered point cloud sequence based on point cloud density analysis to generate a missing region mask.

[0128] In this step, the inter-frame dynamic registration process employs the incremental dynamic registration algorithm described in the system embodiment, which calculates a fast point feature histogram descriptor and iteratively optimizes the registration transformation matrix. The registration convergence threshold and the maximum number of iterations are set in the same way as in the system embodiment. Missing region detection uses the adaptive missing region detection algorithm described in the system embodiment, achieving automatic detection through voxel mesh generation and density statistics.

[0129] Step S2, adversarial missing region completion step: Receive the registered point cloud sequence and the missing region mask, generate point clouds for the missing regions through a generator network, and determine the authenticity of the generated point clouds through a discriminator network. Iteratively optimize the parameters of the generator network based on adversarial training, and output the completed point cloud.

[0130] In this step, the structures of the generator network and discriminator network are the same as in the system embodiment. The loss function and optimization strategy for adversarial training are also the same as in the system embodiment. This step also receives geometric constraint feedback parameters from step S3, and adjusts the generation strategy according to the feedback parameters to form a closed-loop optimization.

[0131] Step S3, geometric constraint surface reconstruction step: Receive the completed point cloud, perform implicit surface fitting on the completed point cloud based on adversarial geometric constraint loss, generate the reconstructed surface mesh, and feed back the geometric constraint feedback parameters to step S2 to adjust the generation strategy.

[0132] In this step, the implicit surface fitting employs the neural implicit surface representation method and adversarial geometric constraint loss function described in the system embodiments. This step also receives topology optimization feedback parameters from step S5, and adjusts the surface fitting parameters based on these feedback parameters to form a second closed-loop optimization.

[0133] Step S4, multi-view feature fusion step: receive the reconstructed surface mesh, perform multi-view projection on the reconstructed surface mesh to obtain a multi-view depth map, perform feature fusion on the multi-view depth map through an attention weighting mechanism, back-project the fused features onto the reconstructed surface mesh, and output the fused enhanced mesh.

[0134] In this step, the specific implementations of multi-view projection, feature extraction, attention fusion, and back projection are the same as in the system implementation. The multi-view attention fusion algorithm calculates the feature correlation between views and assigns fusion weights through a self-attention mechanism.

[0135] Step S5, Topology Optimization and Intelligent Matching: Receive the fused enhanced mesh, perform mesh simplification and topology repair on the fused enhanced mesh, calculate the topology consistency index, feed back the topology optimization feedback parameters to step S3 to adjust the surface fitting parameters, output the final output model, and perform treatment plan matching based on the tooth morphology knowledge base to generate a recommended list of treatment plans.

[0136] In this step, the specific implementations of mesh simplification, topology repair, and topology consistency index calculation are the same as in the system embodiment. The adaptive topology optimization algorithm uses a quadratic error metric method for mesh simplification. Treatment plan matching employs the treatment plan similarity matching algorithm described in the system embodiment, calculating morphological similarity based on Hausdorff distance and chamfer distance.

[0137] Through the coordinated processing of the above five steps, the method of this invention achieves incremental acquisition of oral scan data, adversarial completion of missing areas, geometric constraint reconstruction of tooth surfaces, attentional fusion of multi-view features, and intelligent matching and recommendation of treatment plans, forming an intelligent processing process from scanning to diagnosis. Steps S2 and S3 form a first closed loop through geometric constraint feedback parameters, and steps S3 and S5 form a second closed loop through topology optimization feedback parameters. This dual closed-loop structure enables the system to dynamically adjust upstream processing strategies based on downstream processing results, significantly improving reconstruction quality and system robustness.

[0138] To verify the effectiveness of the technical solution of this invention, experimental verification was conducted on a test dataset containing 500 oral scan data. The test dataset covers scan cases with different tooth positions, different degrees of tooth loss, and different noise levels.

[0139] Regarding the accuracy of missing region completion, the average chamfer distance between the completed region and the real data in this invention is 0.08mm, which is 55% lower than the 0.18mm of the traditional linear interpolation method and 43% lower than the 0.14mm of the data augmentation method in Chinese invention CN117252847A.

[0140] Regarding surface reconstruction accuracy, the average Hausdorff distance between the reconstructed surface and the real surface using the method of this invention is 0.12 mm, meeting the accuracy requirements for clinical applications. The restoration rate of fine structures such as occlusal grooves and fissures reaches 92%, significantly higher than the 65% of traditional methods.

[0141] In terms of processing efficiency, the incremental processing mode of this invention achieves a processing latency of less than 50ms per frame and supports real-time feedback during the scanning process. The reconstruction time of the complete dental arch is approximately 3 minutes, which is 70% shorter than the 10 minutes of traditional offline processing methods.

[0142] Regarding the accuracy of treatment plan matching, clinical experts have assessed that at least one of the top three recommended treatment plans in this invention was adopted in 87% of cases, demonstrating the practical value of the treatment plan matching function.

[0143] The embodiments described above are merely illustrative of specific implementations of the present invention, and while the descriptions are detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. An incremental adversarial reconstruction system for three-dimensional oral cavity scan data, characterized in that, include: An incremental scanning data acquisition module is used to receive the original scanning point cloud output by the oral scanner, perform inter-frame dynamic registration processing on the original scanning point cloud to obtain a registered point cloud sequence, and detect missing regions in the registered point cloud sequence based on point cloud density analysis to generate a missing region mask. An adversarial missing region completion module is used to receive the registered point cloud sequence and the missing region mask, generate point clouds for the missing regions through a generator network, and determine the authenticity of the generated point clouds through a discriminator network. The parameters of the generator network are iteratively optimized based on adversarial training, and the completed point cloud is output. The geometric constraint surface reconstruction module is used to receive the completed point cloud, perform implicit surface fitting on the completed point cloud based on adversarial geometric constraint loss, generate a reconstructed surface mesh, and send the geometric constraint feedback parameters to the adversarial missing region completion module to adjust the generation strategy of the generator network. A multi-view feature fusion module is used to receive the reconstructed surface mesh, perform multi-view projection on the reconstructed surface mesh to obtain a multi-view depth map, perform feature fusion on the multi-view depth map through an attention weighting mechanism, back-project the fused features onto the reconstructed surface mesh, and output a fused enhanced mesh. The topology optimization and intelligent matching module is used to receive the fused enhanced mesh, perform mesh simplification and topology repair processing on the fused enhanced mesh, calculate the topology consistency index, send the topology optimization feedback parameters to the geometric constraint surface reconstruction module to adjust the surface fitting parameters, output the final output model, and perform treatment plan matching on the final output model based on the tooth morphology knowledge base to generate a treatment plan recommendation list.

2. The system according to claim 1, characterized in that, In the incremental scanning data acquisition module, the inter-frame dynamic registration processing includes: setting the registration convergence threshold to 0.01mm to 0.1mm and setting the maximum number of iterations to 50 to 200; in the point cloud density analysis, regions with local point cloud density lower than a preset density threshold are marked as missing regions, and the preset density threshold is 0.3 to 0.5 times the average point cloud density.

3. The system according to claim 1, characterized in that, In the adversarial missing region completion module, the generator network includes an encoder and a decoder. The encoder includes 4 to 8 convolutional layers, and the decoder includes 4 to 8 deconvolutional layers. The discriminator network includes 5 to 10 convolutional layers. The learning rate for adversarial training is set to 0.0001 to 0.001, and the number of training iterations is set to 1000 to 10000.

4. The system according to claim 1, characterized in that, In the geometrically constrained surface reconstruction module, the adversarial geometrical constraint loss includes a surface smoothness constraint term, a normal consistency constraint term, and a curvature continuity constraint term, and the sum of the weight coefficients of the three constraint terms is 1; the implicit surface fitting adopts a signed distance field representation, and the mesh resolution is set to 64 to 256.

5. The system according to claim 1, characterized in that, In the adversarial missing region completion module, the generator network is based on a conditional generative adversarial network architecture, using the missing region mask and the neighboring point clouds in the registered point cloud sequence as conditional inputs to generate a completed point cloud that is geometrically continuous with the neighboring point clouds; the discriminator network adopts a multi-scale discrimination strategy to perform authenticity discrimination at the local neighborhood scale and the global dental arch scale respectively.

6. The system according to claim 1, characterized in that, In the geometrically constrained surface reconstruction module, the adversarial geometric constraint loss is calculated by an auxiliary discriminator network. The auxiliary discriminator network takes the local surface patches of the reconstructed surface mesh as input and determines whether the local surface patches conform to the anatomical features of teeth. The geometric constraint feedback parameters include local curvature deviation value and normal deviation angle value.

7. The system according to claim 1, characterized in that, In the multi-view feature fusion module, the multi-view projection includes 6 to 12 uniformly distributed view directions; the attention weighting mechanism is a self-attention mechanism, which assigns fusion weights to each view by calculating the feature correlation between depth maps of different views; the back projection uses a differentiable rendering method to map two-dimensional features back to a three-dimensional surface.

8. The system according to claim 1, characterized in that, In the topology optimization and intelligent matching module, the mesh simplification adopts a quadratic error measurement method to reduce the number of mesh patches to 10% to 50% of the original number; the topology repair process includes hole filling, non-manifold edge repair, and self-crossing detection and elimination; the topology consistency index includes Euler characteristic number and genus number.

9. The system according to claim 1, characterized in that, In the topology optimization and intelligent matching module, the tooth morphology knowledge base includes a standard tooth anatomy template library and a historical treatment plan database; the treatment plan matching is based on morphological similarity calculation, which is obtained by calculating the Hausdorff distance and chamfer distance between the final output model and the templates in the standard tooth anatomy template library; the treatment plan recommendation list is arranged in descending order of similarity score, and the top 3 to 10 treatment plans with the highest similarity scores are output.

10. An incremental adversarial reconstruction method for three-dimensional oral cavity scan data, using the system described in claims 1-9, characterized in that, include: The incremental scanning data acquisition step involves receiving the original scan point cloud output by the oral scanner, performing inter-frame dynamic registration processing on the original scan point cloud to obtain a registered point cloud sequence, and detecting missing regions in the registered point cloud sequence based on point cloud density analysis to generate a missing region mask. The adversarial missing region completion step involves receiving the registered point cloud sequence and the missing region mask, generating point clouds from the missing regions using a generator network, judging the authenticity of the generated point clouds using a discriminator network, iteratively optimizing the parameters of the generator network based on adversarial training, and outputting the completed point cloud. The geometric constraint surface reconstruction step receives the completed point cloud, performs implicit surface fitting on the completed point cloud based on adversarial geometric constraint loss, generates a reconstructed surface mesh, and feeds back the geometric constraint feedback parameters to the adversarial missing region completion step to adjust the generation strategy. The multi-view feature fusion step involves receiving the reconstructed surface mesh, performing multi-view projection on the reconstructed surface mesh to obtain a multi-view depth map, performing feature fusion on the multi-view depth map through an attention weighting mechanism, back-projecting the fused features onto the reconstructed surface mesh, and outputting a fused enhanced mesh. The topology optimization and intelligent matching steps involve receiving the fused enhanced mesh, performing mesh simplification and topology repair on the fused enhanced mesh, calculating the topology consistency index, feeding back the topology optimization feedback parameters to the geometric constraint surface reconstruction step to adjust the surface fitting parameters, outputting the final output model, and matching treatment plans based on the tooth morphology knowledge base to generate a recommended list of treatment plans.

Images