Tooth three-dimensional modeling system based on computer vision, computer equipment and readable storage medium

Abstract

The invention relates to the technical field of tooth modeling, and discloses a three-dimensional tooth modeling system based on computer vision, computer equipment and a readable storage medium. According to the method, mirror reflection, diffuse reflection and subsurface scattering components in an original image are separated, mirror reflection intensity is normalized in combination with a dynamic truncation algorithm, pixel saturation is eliminated, groove and nest textures are reserved, a complete point cloud is obtained based on a two-dimensional texture image and cubic spline repair, and a multi-exposure point cloud sequence is obtained through bimodal calibration. The method comprises the following steps: solving the problem of data dislocation, carrying out weight assignment and data fusion on three-dimensional points in a plurality of exposure point cloud sequences to obtain three-dimensional fusion feature data, combining layered optical modeling and photon tracking compensation deviation, fusing clinical constraints, finally dynamically adjusting parameters, feeding back and optimizing, and outputting a micron-sized precision model. The modeling defect caused by difficulty in effectively coordinating feature contribution degrees under different exposure conditions is overcome, and high-precision modeling is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of tooth modeling, in particular to a tooth three-dimensional modeling system based on computer vision, a computer device and a readable storage medium. BACKGROUND

[0002] Tooth three-dimensional modeling is a process of converting two-dimensional images or point cloud data of teeth into accurate three-dimensional digital models using computer vision, optical scanning and other technologies. It collects optical features, spatial coordinates and other multi-dimensional information of tooth surfaces, and constructs a digital model containing crown shape, occlusion relationship, groove structure and other details after data fusion and optimization processing, which can restore the geometric shape and surface characteristics of teeth. In clinical diagnosis and treatment, it provides intuitive basis for caries diagnosis, orthodontic treatment plan design and restoration production, and improves treatment accuracy. In the field of denture processing, it realizes digital production, shortens the production cycle and improves the adaptability. At the same time, it builds a visual platform for oral medicine teaching, doctor-patient communication and helps patients understand treatment plans.

[0003] The existing tooth three-dimensional modeling system cannot effectively coordinate the feature contribution under different exposure conditions, and it is difficult to optimize the key anatomical structures, so that the key structural features of the teeth are incomplete, resulting in insufficient construction accuracy of the tooth three-dimensional model. SUMMARY

[0004] The main purpose of the present application is to provide a tooth three-dimensional modeling system based on computer vision, a computer device and a readable storage medium, which aims to solve the technical problems in the prior art.

[0005] The present application provides a tooth three-dimensional modeling system based on computer vision, comprising:

[0006] A data acquisition module is configured to acquire tooth multi-mode core data based on computer vision, wherein the multi-mode core data includes original image data and camera calibration data.

[0007] A texture generation module is configured to acquire a two-dimensional texture image based on the original image data.

[0008] A point cloud reconstruction module is configured to acquire a projected original image based on the two-dimensional texture image, and acquire a complete point cloud data set based on the projected original image and the camera calibration data.

[0009] A sequence acquisition module is configured to acquire a plurality of exposure point cloud sequences based on the camera calibration data, the two-dimensional texture image and the complete point cloud data set.

[0010] A feature fusion module is configured to fuse a plurality of the exposure point cloud sequences to obtain three-dimensional fusion feature data.

[0011] a first optimization module, configured to construct an initial optimization dataset according to the three-dimensional fusion feature data;

[0012] a second optimization module, configured to construct a final optimization dataset according to the initial optimization dataset;

[0013] a model construction module, configured to construct a three-dimensional tooth model according to the final optimization dataset.

[0014] Preferably, the texture generation module comprises:

[0015] a component extraction unit, configured to acquire a specular reflection reference image, a diffuse reflection background image and a standard brightness image according to the original image data, and acquire a specular reflection component, a diffuse reflection component and a subsurface scattering component according to the specular reflection reference image and the diffuse reflection background image;

[0016] a coefficient solving unit, configured to acquire a target constraint condition, and acquire a specular reflection coefficient, a diffuse reflection coefficient and a subsurface scattering coefficient according to the specular reflection component, the diffuse reflection component, the subsurface scattering component and the target constraint condition;

[0017] a first acquisition unit, configured to acquire a specular reflection relative intensity of a plurality of pixel points according to the specular reflection coefficient, the diffuse reflection coefficient and the subsurface scattering coefficient, and acquire a specular reflection intensity map according to the specular reflection relative intensity of each pixel point;

[0018] a reflection alignment unit, configured to acquire a reflection feature alignment map according to the standard brightness image and the specular reflection intensity map, and acquire a diffuse reflection proportion map according to the reflection feature alignment map;

[0019] a texture synthesis unit, configured to acquire a two-dimensional texture image according to the diffuse reflection proportion map, the specular reflection reference image and the diffuse reflection background image.

[0020] Preferably, the point cloud reconstruction module comprises:

[0021] a feature extraction unit, configured to acquire a gray level histogram and an enamel pit mean width according to the two-dimensional texture image, and acquire a high reflection area coordinate according to the gray level histogram;

[0022] a projection generation unit, configured to acquire a fundamental frequency, a second harmonic and a third harmonic according to the enamel pit mean width, and acquire a projection original image according to the fundamental frequency, the second harmonic and the third harmonic;

[0023] a projection correction unit, configured to acquire a basic compensation amount according to the camera calibration data, and acquire a projection corrected image according to the basic compensation amount and the projection original image;

[0024] An exposure selection unit is configured to acquire an enamel area ratio and a saturated pixel ratio according to the projection correction image, and acquire a multi-exposure sequence image set according to the enamel area ratio;

[0025] A three-dimensional initial construction unit is configured to acquire a three-dimensional point cloud set according to the multi-exposure sequence image set;

[0026] A point cloud integration unit is configured to acquire an exposure time-depth error correspondence table, and acquire a complete point cloud data set according to the exposure time-depth error correspondence table, the three-dimensional point cloud set and the saturated pixel ratio.

[0027] Preferably, the sequence acquisition module comprises:

[0028] A parameter calibration unit is configured to acquire intrinsic parameters of a calibration board, initial calibration parameters and an initial extrinsic parameter matrix according to the camera calibration data, and acquire three-dimensional coordinates of feature points according to the intrinsic parameters of the calibration board;

[0029] A feature detection unit is configured to acquire two-dimensional coordinates of feature points and effective pixel brightness according to the two-dimensional texture image;

[0030] A deviation acquisition unit is configured to acquire theoretical projection coordinates according to the intrinsic parameters of the calibration board and the initial calibration parameters, and acquire pixel coordinate deviations according to the theoretical projection coordinates and the two-dimensional coordinates of the feature points;

[0031] An extrinsic parameter optimization unit is configured to acquire an optimized extrinsic parameter matrix according to the initial extrinsic parameter matrix and the pixel coordinate deviations, and acquire normal vectors of the feature points according to the optimized extrinsic parameter matrix and the three-dimensional coordinates of the feature points;

[0032] A sequence generation unit is configured to acquire a brightness-normal vector mapping rule according to the effective pixel brightness and the normal vectors of the feature points, and acquire a multi-exposure point cloud sequence according to the complete point cloud data set, the brightness-normal vector mapping rule and the optimized extrinsic parameter matrix.

[0033] Preferably, the feature fusion module comprises:

[0034] A parameter extraction unit is configured to extract feature parameters of each three-dimensional point in a plurality of the exposure point cloud sequences, wherein the feature parameters comprise an exposure time, a brightness value, a gradient amplitude, a Gaussian curvature and a reflection probability;

[0035] A weight construction unit is configured to respectively assign weights to the feature parameters to obtain a feature weight set containing weight information;

[0036] A coordinate fusion unit is configured to acquire a preset registration point cloud sequence, and acquire final fusion coordinates of each three-dimensional point according to the registration point cloud sequence, the plurality of exposure point cloud sequences and the feature weight set.

[0037] a normal correction unit configured to obtain an anti-symmetric matrix and an original normal vector of each three-dimensional point, and obtain a rotation compensation matrix according to the anti-symmetric matrix and the exposure time, and correct the original normal vector according to the rotation compensation matrix to obtain a final normal vector;

[0038] a feature integration unit configured to integrate the final fused coordinate and the final normal vector to obtain three-dimensional fused feature data.

[0039] Preferably, the first optimization module comprises:

[0040] a distance calculation unit configured to extract a Gaussian curvature in the three-dimensional fused feature data, and calculate a geodesic distance from each three-dimensional point to a nearest edge point according to the Gaussian curvature to obtain a geodesic distance set;

[0041] a scattering compensation unit configured to obtain a preset scattering compensation coefficient set, and obtain an enamel scattering compensation coordinate of each three-dimensional point according to the scattering compensation coefficient set and the geodesic distance set;

[0042] a neighborhood analysis unit configured to obtain a plurality of three-dimensional point radius neighborhoods and corresponding neighborhood normal vector gradients according to the enamel scattering compensation coordinate, and obtain a shape index and a shape index gradient corresponding to each three-dimensional point radius neighborhood according to the neighborhood normal vector gradient;

[0043] a pit screening unit configured to screen the three-dimensional point radius neighborhood according to the shape index to obtain a deep pit region, and obtain a conduction coefficient corresponding to the deep pit region according to the shape index gradient;

[0044] an initial optimization unit configured to adjust an anisotropic diffusion filter according to the conduction coefficient to obtain pit region point cloud data, and construct an initial optimization data set according to the pit region point cloud data and the enamel scattering compensation coordinate.

[0045] Preferably, the second optimization module comprises:

[0046] a hierarchical division unit configured to obtain a vertical distance of each three-dimensional point according to the three-dimensional fused feature data, and divide each three-dimensional point according to the vertical distance to obtain layered structure information;

[0047] an optical modeling unit configured to obtain a scattering coefficient set according to the layered structure information and the vertical distance, and obtain a structured optical parameter field according to the scattering coefficient set, the layered structure information, and the initial optimization data set;

[0048] a photon distribution unit configured to obtain a region division type according to the feature parameters and the structured optical parameter field, and obtain a photon emission angle and a photon distribution number according to the region division type;

[0049] a path calculation unit configured to obtain a photon motion path according to the photon emission angle and the photon distribution number, and obtain a deviation compensation amount according to the photon motion path;

[0050] a final optimization unit configured to obtain compensation constraint data according to the initial optimization data set, and obtain a final optimization data set according to the compensation constraint data, the deviation compensation amount and the structured optical parameter field.

[0051] Preferably, the model construction module comprises:

[0052] a model initial construction unit configured to obtain simplified point cloud data, curvature enhancement weight and core configuration parameters according to the final optimization data set, and obtain a preliminary three-dimensional model according to the core configuration parameters, the curvature enhancement weight and the simplified point cloud data;

[0053] a model processing unit configured to obtain a plurality of vertex radius neighborhoods and corresponding vertex densities according to the preliminary three-dimensional model, and screen a plurality of the vertex radius neighborhoods according to the vertex densities to obtain a topological optimization model;

[0054] a groove depth obtaining unit configured to obtain a groove region depth according to the topological optimization model;

[0055] a parameter optimization unit configured to obtain a groove basic depth, obtain a groove depth error according to the groove basic depth and the groove region depth, and adjust the core configuration parameters according to the groove depth error to obtain optimized configuration parameters;

[0056] a final mold generation unit configured to obtain a tooth three-dimensional model according to the optimized configuration parameters, the curvature enhancement weight and the simplified point cloud data.

[0057] The application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the memory is configured to manage the operation of the computer vision-based tooth three-dimensional modeling system when the computer program is executed.

[0058] The application further provides a computer readable storage medium, which stores a computer program, and the computer program is configured to manage the operation of the computer vision-based tooth three-dimensional modeling system when the computer program is executed by a processor.

[0059] The present application has the beneficial effects that: the present application obtains multi-mode core data, comprehensively collects tooth modeling basic information, solves the problem of "loss of texture information caused by reflection", separates the specular reflection, diffuse reflection and subsurface scattering components in the original image, normalizes the specular reflection intensity combined with the dynamic truncation algorithm, eliminates pixel saturation and retains the pit texture, obtains the average width of enamel pits based on the two-dimensional texture image to optimize the projection fringe frequency, controls the saturated pixels within 5% by adjusting the projection intensity in real time, combines the cubic B-spline to repair the saturated area, obtains the complete point cloud dataset, solves the problems of "inappropriate projection parameters and incomplete point cloud", then obtains the optimized extrinsic parameter matrix and brightness-normal mapping rule through the dual-mode calibration based on the camera calibration data and two-dimensional texture image, combines the complete point cloud to obtain the multi-exposure point cloud sequence, which can effectively solve the problems of "spatial misplacement of multi-exposure data and mismatch of characteristics" existing in the prior art, and for the roughness problem of multi-exposure point cloud fusion, constructs the optical fusion weight field through the feature parameters, weightedly fuses the coordinates and corrects the normal vector, preferentially uses short exposure data in the highlight area and long exposure data in the dark area to obtain three-dimensional fusion feature data, dynamically corrects the enamel scattering depth error based on this, optimizes the pit point cloud combined with the anisotropic filtering, compensates the scattering deviation through the hierarchical optical modeling and photon tracing simulation, simultaneously integrates the clinical constraints to ensure that the model fits the physiological characteristics, finally outputs the tooth three-dimensional model with micron-level precision through dynamic adjustment of the core parameters and combination of the pit depth error feedback optimization, so as to solve the modeling defects caused by the fact that the prior art cannot effectively coordinate the feature contribution degree under different exposure conditions, and finally realize high-precision tooth three-dimensional modeling. BRIEF DESCRIPTION OF DRAWINGS

[0060] Figure 1 It is a system structure schematic diagram of an embodiment of the present application.

[0061] Figure 2 It is a computer device internal structure schematic diagram of an embodiment of the present application.

[0062] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0063] It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0064] As Figure 1 shown, the present application provides a tooth three-dimensional modeling system based on computer vision. Referring to Figure 1 , in the modeling system,

[0065] a data acquisition module configured to acquire tooth multi-modal core data based on computer vision, wherein the multi-modal core data comprises raw image data and camera calibration data;

[0066] a texture generation module configured to acquire a two-dimensional texture image based on the raw image data;

[0067] a point cloud reconstruction module configured to acquire a projected raw image based on the two-dimensional texture image, and acquire a complete point cloud data set based on the projected raw image and the camera calibration data;

[0068] a sequence acquisition module configured to acquire a plurality of exposure point cloud sequences based on the camera calibration data, the two-dimensional texture image, and the complete point cloud data set;

[0069] a feature fusion module configured to fuse the plurality of exposure point cloud sequences to obtain three-dimensional fused feature data;

[0070] a first optimization module configured to construct an initial optimization data set based on the three-dimensional fused feature data;

[0071] a second optimization module configured to construct a final optimization data set based on the initial optimization data set;

[0072] a model construction module configured to construct a tooth three-dimensional model based on the final optimization data set.

[0073] As the function of each module described above, the present application realizes the comprehensive collection of the basic information of tooth modeling by acquiring multi-mode core data, wherein the multi-mode core data refers to a set of core basic data obtained through various ways or channels for supporting the whole process of three-dimensional model construction, including original image data and camera calibration data, wherein the original image data refers to the unprocessed image information obtained by a camera or other image acquisition device, and the camera calibration data refers to the data obtained by a specific calibration method before the camera is used, which is used to describe the internal parameters (such as focal length, principal point coordinates, etc.) and external parameters (such as the position and attitude of the camera in the three-dimensional space) of the camera. Then, by separating the specular reflection component, diffuse reflection component and subsurface scattering component in the original image, and combining the dynamic truncation algorithm to normalize the specular reflection intensity, the local pixel saturation phenomenon is eliminated, and the original texture details of the occlusal surface groove are retained, so as to obtain a two-dimensional texture image, wherein the two-dimensional texture image refers to a two-dimensional image obtained by extracting and processing the original image data to represent the texture information of the surface of the target object, so as to solve the problem of "loss of texture information caused by reflection" existing in the prior art. Then, based on the two-dimensional texture image, the uniform width of the enamel groove is obtained and the projection stripe frequency is optimized, and further, a projection original image is obtained, wherein the projection original image refers to an image obtained by projecting a three-frequency composite stripe onto the tooth surface through hardware, while the projection intensity is adjusted in real time to reduce the proportion of saturated pixels (controlled within 5%), and the saturated area is repaired by a cubic B-spline, so as to obtain a complete point cloud data set, thereby reducing the interference of reflection on three-dimensional coordinate acquisition from the source, wherein the complete point cloud data set refers to a set of points obtained by processing the point cloud reconstruction module, which can completely describe the three-dimensional shape of the target object. By this method, the problems of "inappropriate projection parameters and incomplete point cloud" existing in the prior art can be solved. Then, according to the camera calibration data and the two-dimensional texture image, the optimized external parameter matrix and the brightness-normal mapping rule are obtained through the dual-mode calibration, and a plurality of exposure point cloud sequences are obtained in combination with the complete point cloud data set, wherein the exposure point cloud sequence refers to a series of point cloud data obtained under different exposure conditions, so as to ensure the consistency of the multi-exposure point cloud sequence in geometry and optical characteristics, and to provide accurate data basis for subsequent fusion, so as to solve the problem of "spatial mislocation of multi-exposure data and mismatch of characteristics" existing in the prior art.

[0074] Due to the prior art using simple average fusion for multi-exposure point clouds, without considering differences such as reflection and curvature, it is difficult to effectively coordinate the feature contribution under different exposure conditions, thus there are problems of low accuracy in the pit and furrow area and deflection of the normal vector, and the method of fusing multiple exposure point cloud sequences to obtain three-dimensional fused feature data can effectively solve the problems of "rough multi-exposure data fusion and normal distortion", wherein the three-dimensional fused feature data refers to a comprehensive feature set formed by integrating the feature parameters of each three-dimensional point, and we use image data {F1, F2, F3, …, F n ,} to represent the three-dimensional fused feature data, for each F={C, N, S, P,}, wherein C is the final fused coordinates used to represent the spatial position of the three-dimensional point, N is the final normal vector used to represent the corrected three-dimensional point, S is the dominant exposure source identifier used to mark which point cloud data (such as 1ms, 5ms, 10ms, etc.) the final fusion result of the three-dimensional point mainly depends on, and P is used to represent the feature parameters, including the exposure time, brightness value, gradient amplitude, Gaussian curvature and reflection probability of the three-dimensional point. The three-dimensional fused feature data is output in the form of a PLY format file, which not only realizes accurate fusion of multi-exposure data, but also completely retains the original attribute information of the point cloud. The essence of the above method of obtaining three-dimensional fused feature data is to preferentially retain the accurate coordinates of short exposure data in highlight areas, and focus on the detail information of long exposure data in pit dark areas, while correcting the deflection of the normal vector caused by reflection through a rotation compensation matrix to improve the authenticity of three-dimensional features. Based on the three-dimensional fused feature data, the depth error caused by enamel scattering is dynamically corrected, and the point cloud in the pit area is optimized by combining anisotropic diffusion filtering, which smooths internal noise while maintaining edge sharpness, to obtain an initial optimized data set. Through hierarchical optical modeling and photon tracing simulation, the spatial positioning deviation caused by light scattering is quantified and compensated, and clinical anatomical constraints (such as the threshold of the curvature radius of the healthy tooth pit) are integrated to ensure that the model meets the physiological characteristics, to obtain a final optimized data set. The final optimized data set refers to the data set obtained by further optimization and refinement, which further eliminates errors and deviations in the data, and is the final data basis for constructing a three-dimensional tooth model. This can effectively solve the problem of "interlayer scattering error uncorrected and model deviating from physiological characteristics" in the prior art. Finally, through dynamic adjustment of core parameters and the method of feedback optimization of pit depth error, a tooth three-dimensional model with micron-level accuracy is finally output. The tooth three-dimensional model refers to a digital three-dimensional geometric entity constructed by computer vision, optical scanning technology and iterative optimization process. Through the above modeling method, the modeling defects caused by the inability to effectively coordinate the feature contribution under different exposure conditions and the difficulty in targeted optimization of key anatomical structures can be solved, and finally high-precision three-dimensional modeling of teeth is realized.

[0075] In one embodiment, the texture generation module comprises:

[0076] a component extraction unit configured to obtain a specular reference image, a diffuse background image and a standard brightness image according to the original image data, and obtain a specular component, a diffuse component and a subsurface scattering component according to the specular reference image and the diffuse background image;

[0077] a coefficient solving unit configured to obtain a target constraint condition, and obtain a specular coefficient, a diffuse coefficient and a subsurface scattering coefficient according to the specular component, the diffuse component, the subsurface scattering component and the target constraint condition;

[0078] a first obtaining unit configured to obtain a specular relative intensity of a plurality of pixel points according to the specular coefficient, the diffuse coefficient and the subsurface scattering coefficient, and obtain a specular intensity map according to the specular relative intensity of each pixel point;

[0079] a reflection alignment unit configured to obtain a reflection feature alignment map according to the standard brightness image and the specular intensity map, and obtain a diffuse proportion map according to the reflection feature alignment map;

[0080] a texture synthesis unit configured to obtain a two-dimensional texture image according to the diffuse proportion map, the specular reference image and the diffuse background image.

[0081] As the function of each unit described above, the present application obtains a specular reference image, a diffuse background image and a standard brightness image from the original image data, wherein the specular reference image refers to a tooth image collected by a specific polarization direction (such as perpendicular to the polarization direction of incident light), the diffuse background image refers to a tooth image collected orthogonally to the polarization direction of the specular reference image (such as parallel to the polarization direction of incident light), and the standard brightness image refers to a tooth image collected by a conventional non-polarized light source, reflecting the overall brightness distribution of the tooth surface. After obtaining the specular reference image and the diffuse background image, the following calculation is performed for each pixel point: first, the absolute value of the brightness difference of the two images at the point is calculated, then the sum of the brightness of the two images at the point is calculated, and the absolute value of the brightness difference is divided by the sum of the brightness to obtain the local polarization degree (range 0-1) of the point. Then, the local polarization degrees of all pixel points are compared, and the largest value is selected as the maximum polarization degree of the enamel. Then, the specular reflection component is separated by using the strong polarization sensitivity of the specular reflection and combining the maximum polarization degree of the enamel. The specific calculation process is as follows: first, the difference between the specular reference image and the diffuse background image is calculated, and the difference is divided by the maximum polarization degree of the enamel. The maximum value obtained by comparing the result with "0" is the specular reflection component. Secondly, due to the characteristics of the diffuse reflection, such as isotropy and low polarization sensitivity, the minimum value in the specular reference image and the diffuse background image is taken as the initial value of the diffuse reflection component. This operation can naturally avoid the strong reflection area and retain more real diffuse reflection information. Then, the optical diffusion model is used to obtain the initial subsurface scattering component by substituting the initial value of the diffuse reflection component and the specular reflection component. Then, an empirical coefficient (calibrated in the laboratory, usually 0.3) is introduced to subtract the subsurface scattered light mixed with diffuse reflection, thereby obtaining the diffuse reflection component. The specific calculation process is as follows: first, take the minimum value in the specular reference image and the diffuse background image, and calculate the product of the initial subsurface scattering component and the empirical coefficient. Then, the difference between the minimum value and the product is calculated to obtain the diffuse reflection component. Then, the total intensity of the specular reference image and the diffuse background image is calculated, and the twice diffuse reflection component and the specular reflection component are subtracted, and then divided by twice the reduced scattering coefficient to obtain the subsurface scattering component, wherein the reduced scattering coefficient is determined by the product of the scattering coefficient and the difference between "1" and the anisotropy factor. The scattering coefficient is 0.5 mm -1, for describing the scattering ability of the tissue to the light, the anisotropy factor is 0.8, for describing the concentration degree of the scattering direction, the prior art is difficult to comprehensively suppress the optical phenomenon caused by the enamel characteristics, the scheme realizes the accurate separation of the specular reflection component, the diffuse reflection component and the subsurface scattering component, thereby eliminating or reducing the optical interference caused by the high reflection characteristics of the enamel, can effectively process the specular reflection caused by the smooth surface of the enamel and the subsurface scattering caused by the translucency, avoid the local pixel saturation (overexposure) phenomenon in the scanning image, protect the original texture information, reduce the edge blur problem of the light reflection area, and lay a foundation for subsequent accurate measurement and modeling.

[0082] Then the specular reflection coefficient, the diffuse reflection coefficient and the subsurface scattering coefficient need to be obtained, and the specific obtaining method is as follows: first, the average values of the specular reflection reference image and the diffuse reflection background image are calculated to obtain the total reflection intensity, then a target function is set, that is, the error between the result obtained by weighting and combining the specular reflection component, the diffuse reflection component and the subsurface scattering component and the actually measured total reflection intensity is minimized, then the target constraint condition is set, from the physical layer, the energy conservation needs to be met, that is, the sum of the specular reflection coefficient, the diffuse reflection coefficient and the subsurface scattering coefficient is 1, from the physiological layer, according to the enamel optical characteristic parameter lookup table, the three coefficients need to be within a specific range; then, the Lagrange multiplier method is used to construct an augmented target function containing the constraint condition, which involves the matrix composed of the reflection components, the total reflection intensity vector and the coefficient vector, a closed-form solution can be obtained through the augmented target function, that is, the specular reflection coefficient, the diffuse reflection coefficient and the subsurface scattering coefficient are obtained, and the projection processing is performed on the coefficients exceeding the range, that is, they are limited in the corresponding reasonable interval, so that the finally obtained result meets all the constraint conditions, the prior art cannot accurately balance each reflection component when processing the enamel optical phenomenon, and the scheme accurately calculates and reasonably constrains the coefficients, so that the subsequent processing of the reflection component is more scientific and accurate, so as to improve the processing ability of the modeling system to the complex optical characteristics of the enamel.

[0083] Then the sum of the diffuse reflection coefficient and the subsurface scattering coefficient is calculated, and the ratio of 2 times the specular reflection coefficient to the sum is calculated to obtain a normalized reference, then a standardization difference operation is performed on each pixel point: the absolute value of the luminance difference of the specular reflection reference image and the diffuse reflection background image at the point is taken as the numerator, the polarization sensitive component (mainly reflecting the specular reflection) is extracted through the numerator, the luminance sum of the specular reflection reference image and the diffuse reflection background image at the point is taken as the denominator, and a zero constant prevention constant ε = 10 -6The denominator item represents the total reflection intensity, and the ratio of the numerator item to the denominator item is the specular reflection relative intensity of the pixel point. The polarization difference image is obtained according to the specular reflection relative intensity of all pixel points. Then, for the specular reflection relative intensity of all pixel points in the image, if the specular reflection relative intensity of the pixel point is greater than the normalized reference, the normalized reference is taken as the specular reflection relative intensity of the pixel point; if the specular reflection relative intensity of the pixel point is not greater than the normalized reference, the original value is kept. Through this dynamic truncation, the specular reflection relative intensity of all pixel points is limited within the normalized reference range, the normalization representation of the specular reflection intensity is realized, and the specular reflection intensity image is obtained through the specular reflection relative intensity of all pixel points. The specular reflection intensity image refers to the relative proportion of the specular reflection light of each pixel point to the total reflection light. The present scheme limits the specular reflection relative intensity of all pixel points within the normalized reference range through dynamic truncation, can make the reflection intensity data more standardized, and is convenient for subsequent accurate matching and analysis with other image data, so as to solve the problem that the prior art cannot uniformly measure and compare the reflection intensities of different regions.

[0084] Subsequently, the specular reflection intensity image and the standard brightness image are pixel-level aligned through feature point matching (such as the SIFT algorithm), it is ensured that the specular reflection relative intensity and the Y value of each pixel point come from the same position of the tooth, and z-score standardization processing is performed on the specular reflection relative intensity of each pixel point to eliminate the difference in light intensity, so that the network focuses on the reflectance pattern learning. The Y channel is subjected to [0, 1] linear normalization processing, so that the brightness value is compressed to a unified range, avoiding numerical overflow, a reflection feature alignment image is obtained, and the reflection feature alignment image is taken as the input data of the neural network. The feature extraction module in the neural network architecture adopts a 5-layer residual convolutional network, each layer contains Conv3x3, BatchNorm and LeakyReLU (0.2), the residual connection spans 2 convolutional layers to alleviate the gradient disappearance, and a spatial attention unit is embedded in the third layer. The attention map is obtained through 1x1 convolution and sigmoid function after average pooling and maximum pooling of the feature map. In the loss function design, the main loss is the L1 distance between the predicted coefficient map and the true value. The physical constraint loss uses the diffuse reflection coefficient to calculate the square difference between the expected value of the predicted coefficient map and the sum of the diffuse reflection coefficient and the subsurface scattering coefficient. The total loss is 0.8 times the main loss plus 0.2 times the physical constraint loss. During training, a data set containing 500 groups of ex vivo tooth multi-spectral polarization images is constructed, the true diffuse reflection coefficient true value is obtained by an optical measurement device, the Adam optimizer is used, the initial learning rate is set to 3x10 -4And every 50 epochs decay 0.5, batchsize is 16, input patch size is 256*256, real-time inference stage, by converting the FP32 model to INT8 precision to improve 3 times speed, using depth separable convolution to reduce 50% parameter quantity, and with the help of TensorRT deployment to utilize NVIDIA GPU's TensorCore acceleration, finally get the diffuse reflection proportion value of each pixel point, and construct the diffuse reflection proportion map according to the diffuse reflection proportion value of each pixel point.

[0085] Then according to the specular reflection reference image and the diffuse reflection background image, the mean image and the minimum value image are obtained, and the diffuse reflection gray value of each pixel point is calculated through the formula "I diffuse (x, y) = a(x, y) * I mean (x, y) + [1-a(x, y)] * I min (x, y)", wherein, I diffuse (x, y) represents the gray value of the synthesized diffuse reflection image at pixel (x, y), a(x, y) represents the diffuse reflection proportion value of the diffuse reflection proportion map at pixel (x, y), I mean (x, y) represents the gray value of the mean image at pixel (x, y), I min (x, y) represents the gray value of the minimum value image at pixel (x, y), (x, y) represents the two-dimensional coordinates of the pixels in the image, since the specular reflection reference image and the diffuse reflection background image are images of different polarization directions, the specular reflection (strong polarization sensitive) shows large difference in the two, (such as a certain direction may be overexposed, and the other direction may be weak), and the diffuse reflection (low polarization sensitive) shows small difference in the two, the mean image retains the overall brightness and structure, but may contain specular reflection, the minimum value image can naturally filter out strong light of a certain polarization direction, but may lose details due to excessive suppression, therefore, dynamic balance is needed through the diffuse reflection proportion value: when the diffuse reflection proportion value is close to 1, the pixel is closer to the non-reflective area (high diffuse reflection proportion), the formula mainly retains the complete structure and texture of the mean image, when the diffuse reflection proportion value is close to 0, the pixel belongs to the strong reflective area (high specular reflection proportion), the formula mainly retains the minimum value image to suppress the highlight and avoid overexposure, when the diffuse reflection proportion value is in the middle value (such as 0.3-0.7), the two images are proportionally mixed to balance the needs of structure retention and reflection suppression, so that the final output diffuse reflection gray value eliminates overexposure and completely retains the texture information of key structures such as occlusal surface grooves, and a two-dimensional texture image after reflection suppression is constructed according to the diffuse reflection gray value of each pixel point, the method of pixel-level alignment, normalization processing and neural network used in the scheme can accurately obtain the diffuse reflection proportion information, and further generate a high-quality two-dimensional texture image, which eliminates overexposure and completely retains the texture information of key structures such as occlusal surface grooves.

[0086] In one embodiment, the point cloud reconstruction module comprises:

[0087] a feature extraction unit configured to acquire a gray level histogram and an enamel pit mean width according to the two-dimensional texture image, and acquire high reflection region coordinates according to the gray level histogram;

[0088] a projection generation unit configured to acquire a fundamental frequency, a second harmonic and a third harmonic according to the enamel pit mean width, and acquire a projection original image according to the fundamental frequency, the second harmonic and the third harmonic;

[0089] a projection correction unit configured to acquire a basic compensation amount according to the camera calibration data, and acquire a projection correction image according to the basic compensation amount and the projection original image;

[0090] an exposure optimization unit configured to acquire an enamel area proportion and a saturated pixel proportion according to the projection correction image, and acquire a multi-exposure sequence image set according to the enamel area proportion;

[0091] a three-dimensional initial construction unit configured to acquire a three-dimensional point cloud set according to the multi-exposure sequence image set;

[0092] a point cloud integration unit configured to acquire an exposure time-depth error correspondence table, and acquire a complete point cloud data set according to the exposure time-depth error correspondence table, the three-dimensional point cloud set and the saturated pixel proportion.

[0093] As the function of each unit described above, the present application obtains the gray scale histogram of the two-dimensional texture image by calculating the two-dimensional texture image by using Python code (such as the calcHist function of OpenCV), wherein the gray scale histogram refers to the histogram generated by counting the gray scale value distribution of all pixels in the two-dimensional texture image, so as to understand the distribution of different gray scale values in the image; then, based on the gray scale distribution reflected by the gray scale histogram, the adaptive threshold algorithm (such as the adaptiveThreshold of OpenCV) is used to calculate the average gray scale value of the 11*11 pixel neighborhood, and the region coordinates with the average gray scale value greater than 200 (8-bit image) are taken as the high reflection region coordinates, so as to identify the region with high gray scale value (i.e. high reflection region) in the image, provide a basis for subsequent processing of the high reflection region (such as dynamically switching the projection mode, performing phase compensation, etc.), and then obtain the average enamel fossa width, wherein the average enamel fossa width refers to the average width of the enamel fossa (recess structure) on the occlusal surface of the tooth. Since the fundamental frequency must satisfy the condition that "the feature size contains at least 2 pixel periods" (i.e. the frequency is not more than 1 / 2 of the Nyquist frequency corresponding to the feature size), the ratio obtained by dividing "1" by 2 times the average enamel fossa width is taken as the fundamental frequency, so as to ensure that the minimum feature (such as the tooth fossa) of the target surface can be accurately reconstructed, the second harmonic is 2 times the fundamental frequency, and the third harmonic is 3 times the fundamental frequency. Then, real-time fringe synthesis is performed based on the fundamental frequency, the second harmonic and the third harmonic in the FPGA to obtain a three-frequency composite fringe. The three-frequency composite fringe is projected onto the tooth surface through hardware to obtain a projection original image, and the original phase is calculated based on the four-step phase shift method according to the projection original image. Then, the basic compensation amount is obtained through experimental calibration, and the true phase is calculated by adding the original phase and the basic compensation amount. The true phase is substituted into the three-dimensional reconstruction algorithm (such as phase unwrapping and coordinate conversion) to convert the gray scale / phase information in the projection original image into three-dimensional coordinate data of the tooth surface, and finally generate a projection corrected image without distortion, which can be used for clinical analysis (such as orthodontic planning and restoration design). The present application accurately identifies the high reflection region, reasonably sets the projection parameters, and generates a projection corrected image without distortion through the phase compensation and three-dimensional reconstruction algorithm, which provides a reliable basis for subsequent point cloud data generation, and solves the problem that the existing technology is prone to errors in the process of fringe projection and phase calculation, resulting in distortion of the finally generated image.

[0094] The prior art is difficult to dynamically adjust the exposure time according to the actual situation of the teeth when acquiring point cloud data, resulting in poor data acquisition effect in different areas, so the scheme calculates the proportion of enamel area in the texture image according to the projection correction image through Python code to obtain the enamel area proportion, obtains the enamel area proportion-exposure sequence mapping table, and then acquires the exposure time sequence corresponding to the enamel area proportion according to the enamel area proportion-exposure sequence mapping table, for example: when the proportion is less than 15%, a three-level exposure sequence of [1, 3, 7] milliseconds is adopted, when the proportion is 15%-30%, a [0.5, 2, 5] millisecond sequence is adopted, and when the proportion is greater than 30%, a [0.3, 1, 3] millisecond sequence is adopted, and then the exposure time control is realized through hardware-level pulse width modulation according to the exposure time sequence, the time control accuracy is ±5 microseconds, and the exposure switching response time is not more than 150 microseconds. For the same tooth area, multiple images with different exposure times are collected in a very short time to obtain a multi-exposure sequence image set. Based on the multi-exposure sequence image set, each multi-exposure sequence image corresponding three-dimensional point cloud set is generated through three-dimensional reconstruction algorithm (such as phase unwrapping, coordinate conversion), for example, pc1 corresponds to the reconstruction result of 1ms exposure image, pc2 corresponds to the result of 5ms exposure, and each reflects the tooth surface information under different exposure. Then, the exposure time-depth error corresponding table is obtained, and the depth error weight corresponding to each point cloud set is obtained according to the exposure time-depth error corresponding table. Finally, the point cloud coordinate set is obtained through the weighted fusion algorithm according to the three-dimensional point cloud set and the corresponding depth error weight. Through this acquisition method, the accuracy and integrity of the point cloud data can be effectively improved, and the tooth surface information can be fully reflected.

[0095] And because the prior art cannot monitor and process the pixel saturation problem caused by reflection in real time, there are a lot of missing and error information in the collected images and generated point cloud data, so the projection correction image is input into the FPGA module in the form of data stream, multiple processing units in the module can judge different pixels (whether ≥250) at the same time, and the number of saturated pixels is accumulated synchronously. At the same time, another parallel unit counts the total number of image pixels and calculates the ratio of the number of saturated pixels to the total number of image pixels to obtain the saturation pixel proportion. The whole process does not need to be processed one by one in pixel order, so it can realize real-time statistics at the microsecond level. When the saturation pixel proportion is greater than 5%, the initial projection intensity of the three-frequency composite fringe is first acquired, and the projection adjustment intensity is calculated by the formula , wherein I p represents the projection adjustment intensity, I0 represents the initial projection intensity, and δ represents the saturation pixel proportion. The core of the formula is to dynamically reduce the projection intensity according to the saturation degree of the image, so as to reduce the saturated pixels caused by reflection. It is essentially a feedback type light intensity compensation mechanism. The item is a quantitative attenuation factor of saturation degree. When the proportion of saturated pixels increases, the projection adjustment intensity will be reduced, which means that the more serious the saturation is, the more the projection light intensity is weakened, thereby reducing the reflection of the tooth surface, reducing the proportion of saturated pixels in the newly collected image from the source, adjusting the three-frequency composite stripes by inserting a 120 s delay and adjusting the projection adjustment intensity, and for the same reason, the adjusted three-frequency composite stripes are projected onto the tooth surface through hardware again to obtain the optimized projection original image, and the optimized projection original image is returned to the step of "obtaining the basic compensation amount according to the camera calibration data, and obtaining the projection corrected image according to the basic compensation amount and the projection original image", until the proportion of saturated pixels is not greater than 5%. Through the embedded system to adjust the projection intensity, the essence is to reduce the reflection by reducing the projection light intensity, so that the saturated pixels in the subsequent collected image are reduced. When the proportion of saturated pixels is not greater than 5%, for the missing three-dimensional points in the point cloud caused by saturation, the GPU accelerated cubic B-spline repair is realized through Python. Specifically, in the point cloud repair, the algorithm will first use the effective point cloud data in the non-saturated area to fit a parameterized surface through the cubic B-spline algorithm (i.e. calculate the mathematical expression and control points of the surface according to the coordinates of the known points), then for the missing or incorrect three-dimensional point positions in the saturated area, the interpolated calculation is carried out based on the fitted surface to generate the three-dimensional point coordinates to fill these areas, thereby realizing the repair of the saturated area, and finally outputting the complete point cloud data set after dynamic compensation and repair. Through this method of adjusting the projection intensity in real time, reducing reflection and saturated pixels, and effectively repairing the saturated area, a complete and accurate point cloud data set is generated.

[0096] In one embodiment, the sequence acquisition module comprises:

[0097] A parameter calibration unit is configured to acquire intrinsic parameters of the calibration board, initial calibration parameters and an initial extrinsic parameter matrix according to the camera calibration data, and acquire three-dimensional coordinates of feature points according to the intrinsic parameters of the calibration board.

[0098] A feature detection unit is configured to acquire two-dimensional coordinates of feature points and effective pixel brightness according to the two-dimensional texture image.

[0099] A deviation acquisition unit is configured to acquire theoretical projection coordinates according to the intrinsic parameters of the calibration board and the initial calibration parameters, and acquire pixel coordinate deviations according to the theoretical projection coordinates and the two-dimensional coordinates of the feature points.

[0100] An extrinsic parameter optimization unit is configured to acquire an optimized extrinsic parameter matrix according to the initial extrinsic parameter matrix and the pixel coordinate deviations, and acquire normal vectors of feature points according to the optimized extrinsic parameter matrix and the three-dimensional coordinates of the feature points.

[0101] A sequence generation unit is configured to obtain a brightness-normal mapping rule according to the effective pixel brightness and the feature point normal, and to obtain a multi-exposure point cloud sequence according to the complete point cloud data set, the brightness-normal mapping rule and the optimized extrinsic parameter matrix.

[0102] According to the functions of the above units, the application obtains intrinsic parameters of the calibration board, initial calibration parameters and an initial extrinsic parameter matrix according to camera calibration data, wherein the intrinsic parameters of the calibration board refer to physical attribute parameters inherent to the calibration board, and the initial extrinsic parameter matrix refers to an initial matrix describing the relative pose (rotation and translation) between the camera coordinate system and the world coordinate system, which provides a reference for subsequent calibration. Then, geometric calibration in the dual-mode calibration is performed, feature point two-dimensional coordinates are obtained according to a two-dimensional texture image, and feature point three-dimensional coordinates are obtained according to the intrinsic parameters of the calibration board, wherein the feature point two-dimensional coordinates refer to pixel coordinates of the feature points (such as corner points) of the calibration board in the image coordinate system detected from the two-dimensional texture image, and the feature point three-dimensional coordinates refer to three-dimensional space coordinates of the feature points (such as corner points) of the calibration board determined in the world coordinate system based on the intrinsic parameters of the calibration board. Then, based on the Tsai two-step method, theoretical projection coordinates are obtained according to the intrinsic parameters of the calibration board and the initial calibration parameters, pixel coordinate deviations are obtained according to the theoretical projection coordinates and the feature point two-dimensional coordinates, and the optimized extrinsic parameter matrix adapted to the current scene is finally obtained by iteratively adjusting the initial extrinsic parameter matrix with the feature point three-dimensional coordinates as a reference. Subsequently, optical calibration in the dual-mode calibration is performed, and effective pixel brightness is obtained according to the two-dimensional texture image, wherein the effective pixel brightness refers to the brightness value of the effective pixels retained after eliminating overexposed (saturated) and underexposed pixels in the two-dimensional texture image. Based on the geometric calibration, the feature point normal is obtained according to the optimized extrinsic parameter matrix and the feature point three-dimensional coordinates, wherein the feature point normal refers to the surface normal direction (unit vector) of the feature point calculated based on the optimized extrinsic parameter matrix and the feature point three-dimensional coordinates. Then, the least squares fitting fine-tuning parameters are used to minimize the deviation between the calculated brightness and the actual brightness, so as to determine the brightness-normal mapping rule applicable to the current scene, wherein the brightness-normal mapping rule refers to the functional relationship between the effective pixel brightness and the feature point normal obtained by least squares fitting. Through this dual-mode calibration method, accurate geometric and optical parameter support can be provided for subsequent steps, and the spatial accuracy of the structured light scanning and the prediction accuracy of the optical characteristics are ensured.

[0103] The point cloud structure data is constructed from the complete point cloud data set through C++ code, and the point cloud structure data includes three types of core information of vertex coordinates of three-dimensional points, original normal vectors and color information, and the specific construction process is as follows: the depth information (i.e. the distance from the pixel point to the camera) corresponding to each pixel is obtained according to the projection correction image, and a depth map is constructed according to the depth information, then the two-dimensional coordinates and the corresponding depth value of each pixel in the depth map are converted into three-dimensional space coordinates through the inverse transformation of the perspective projection combined with the camera internal parameters (focal length, principal point coordinates) and the optimized external parameter matrix obtained through geometric calibration, and then the three-dimensional space coordinates are converted into microns (1 decimal place is reserved), that is, the vertex coordinates corresponding to each three-dimensional point are obtained, and for the vertex coordinates corresponding to each pixel in the depth map, the system will extract the RGB color value of the corresponding position from the texture image after reflection suppression according to the two-dimensional pixel position, then based on the neighborhood points (selected through the K nearest neighbor algorithm) of each three-dimensional point in the point cloud, the covariance matrix solution is simultaneously performed on multiple vertices on the parallel computing core of the GPU, and the original normal vector of each three-dimensional point is obtained through matrix eigenvalue decomposition (unitized to form an Nx3 unit vector), which supplements the key attributes of the point cloud, and the normal vector directly reflects the orientation of the tooth surface (surface normal), provides geometric basis for subsequent Poisson reconstruction to generate smooth surfaces (improves geometric accuracy), then the point cloud structure data is converted into a binary PLY format file through Python, and the dental three-dimensional data conforming to the DICOM standard is output, a multi-exposure point cloud sequence is obtained, and key parameters such as exposure time are annotated in the header, so that the data can be read by mainstream three-dimensional software, and for the multi-exposure point cloud sequence, the initial point cloud and subsets of different exposure times (such as point clouds corresponding to 1ms, 5ms and 10ms) are stored according to a specific file structure, and a metadata.json metadata file is generated to record exposure parameters (such as exposure time and projection intensity) of each frame, so as to realize data traceability, and through the "fileization" of the point cloud data, the standardized format (PLY) and metadata management, it is convenient for clinical archiving, software analysis and subsequent algorithm calling (such as Poisson reconstruction to generate STL model).

[0104] In one embodiment, the feature fusion module comprises:

[0105] A parameter extraction unit is configured to extract feature parameters of each three-dimensional point in the plurality of exposure point cloud sequences, wherein the feature parameters include exposure time, brightness value, gradient amplitude, Gaussian curvature and reflection probability.

[0106] A weight construction unit is configured to assign weights to the feature parameters respectively to obtain a feature weight set containing weight information.

[0107] a coordinate fusion unit configured to obtain a preset registration point cloud sequence, and obtain a final fusion coordinate of each three-dimensional point according to the registration point cloud sequence, the plurality of exposure point cloud sequences and the feature weight set;

[0108] a normal correction unit configured to obtain an anti-symmetric matrix and an original normal vector of each three-dimensional point, obtain a rotation compensation matrix according to the anti-symmetric matrix and the exposure time, and correct the original normal vector according to the rotation compensation matrix to obtain a final normal vector;

[0109] a feature integration unit configured to integrate the final fusion coordinate and the final normal vector to obtain three-dimensional fusion feature data.

[0110] As the functions of the above units, the application extracts feature parameters of each three-dimensional point in the plurality of exposure point cloud sequences, wherein the feature parameters refer to a parameter set describing the core attributes of each three-dimensional point, including exposure time, brightness value, gradient amplitude, Gaussian curvature and reflection probability, covering geometric features and optical properties, then respectively assigns weights to the feature parameters to obtain a feature weight set containing weight information, wherein the feature weight set refers to a set of single-dimensional weights calculated for each feature parameter of each three-dimensional point in the plurality of exposure point cloud sequences, including brightness time weight, gradient weight, curvature weight and reflection weight, and the specific acquisition method is as follows: since the longer the exposure time, the more light signals the camera receives, the distribution range of the brightness value is usually wider (may contain more extreme values of overexposure or underexposure), based on this logic, first convert the exposure time into a dimensionless multiple with "1 millisecond" as the reference (for example, when the exposure time is 10 ms, the ratio is 10), then substitute it as a real number into the logarithmic operation with "10" as the base, multiply the result by "10", add "35" to the product, and the result is the exposure dynamic adjustment standard deviation, so as to realize the dynamic adjustment of the standard deviation with the exposure time, so that when the exposure time increases, the brightness value is allowed to be accepted in a wider range, avoiding excessive punishment for reasonable brightness fluctuations under long exposure, then calculate the brightness time weight through the formula , wherein ω τ (P) represents the brightness time weight of three-dimensional point P, exp represents the natural exponential function, B(P) represents the brightness value of three-dimensional point P, represents the square of the exposure dynamic adjustment standard deviation of a three-dimensional point P, P represents a certain three-dimensional coordinate point in a multi-exposure point cloud sequence, the essence of the formula is a Gaussian function centered on 128, the closer the brightness value is to 128 (the median brightness), the closer the weight is to 1 (the highest weight), the farther the deviation from 128 (overexposure or underexposure), the smaller the weight (the stronger the penalty), and the adaptive filtering of the brightness data under different exposure conditions is realized by combining the dynamically adjusted exposure dynamic adjustment standard deviation: both the points with moderate brightness and the points with wide brightness distribution in the long exposure scene are considered, and finally the fusion weight of the multi-exposure data is balanced; then the neighborhood 3*3 pixels of each three-dimensional point are obtained, and the horizontal gradient and the vertical gradient in the neighborhood of each three-dimensional point are calculated, and then the gradient amplitude corresponding to each three-dimensional point is calculated according to the horizontal gradient and the vertical gradient by the Pythagorean theorem, and the gradient amplitude reflects the degree of brightness / gray level change in the neighborhood of the three-dimensional point: high gradient amplitude corresponds to sharp edges (such as object outlines and light-dark boundaries) in the image, and low gradient amplitude corresponds to flat areas (such as uniform color blocks), then the ratio of the gradient amplitude to 255 is calculated, and the difference between "1" and the ratio is calculated to obtain the gradient weight of each three-dimensional point, so as to reduce the weight of the high gradient area, thereby reducing the artifacts or distortions at the edges in the subsequent fusion (such as multi-exposure image fusion and point cloud data fusion), and ensuring the sharpness of the edges and the consistency of the fusion results; then the point cloud data in the neighborhood of 0.1mm radius of each three-dimensional point is analyzed by PCA to obtain the Gaussian curvature of each three-dimensional point (positive value represents convex surface, negative value represents concave surface, and the larger the absolute value, the more significant the curvature), and the Gaussian curvature is a key parameter for describing the local geometry of a three-dimensional surface. In the enamel structure, the absolute value of the Gaussian curvature of complex anatomical regions such as grooves and pits is large (the curvature is significant), and the absolute value of the Gaussian curvature of flat regions is small (the curvature is flat), so the curvature weight is calculated by the formula wherein ω g (P) represents the curvature weight of the three-dimensional point P, represents the enamel curvature sensitive coefficient, which is a constant preset by the system (usually 0.05mm -1 ), used to adjust the influence degree of curvature on the weight, and K(P) represents the Gaussian curvature of the three-dimensional point P, P represents a certain three-dimensional coordinate point in a multi-exposure point cloud sequence, and the curvature weight is calculated by the formula The denominator term is constructed, so that the greater the absolute value of the Gaussian curvature (such as the groove area), the greater the denominator, and the smaller the weight, and vice versa, the weight of the flat area is closer to 1. This design can reduce the influence of the high curvature area in data fusion; then the three-dimensional points are projected onto the two-dimensional texture image through a preset projection matrix (the matrix is determined by the external parameter calibration result of the point cloud and the polarization image, and the mapping relationship between the three-dimensional space coordinates and the two-dimensional image pixel coordinates can be established), to obtain the corresponding pixel position of the three-dimensional point in the two-dimensional image. Then, using the existing polarization difference image, the pixel value at the above pixel position is extracted, that is, the reflection probability corresponding to the three-dimensional point is obtained. In three-dimensional point cloud collection (such as optical scanning), reflection will cause the brightness, position and other information of the three-dimensional point to be distorted (for example, metal, smooth object surface is easy to reflect light), and the polarization difference image can distinguish the reflection area and the normal area through the polarization light characteristics. The greater the reflection probability, the more serious the reflection interference of the point, and the lower the data reliability. Therefore, a reflection sensitivity coefficient is further obtained, wherein the reflection sensitivity coefficient is a preset constant (usually 0.5), which is used to adjust the influence amplitude of the reflection probability on the weight, to avoid excessive reduction or insufficient weight due to reflection. The product of the reflection probability corresponding to each three-dimensional point and the reflection sensitivity coefficient is calculated, and the difference between 1 and the product is calculated to obtain the reflection weight of each three-dimensional point. Through this calculation formula, the weight suppression of the reflection area can be realized: when the reflection probability increases (reflection enhancement), the reflection weight decreases, reducing the weight of the point in data fusion; otherwise, the weight of the weak reflection point (small reflection probability) is closer to 1, to retain its effective information. Finally, according to the brightness time weight, gradient weight, curvature weight and reflection weight corresponding to each three-dimensional point, the product of the comprehensive weight corresponding to each three-dimensional point is calculated, and the min-max normalization processing is performed on the comprehensive weight of all three-dimensional points to obtain the point cloud comprehensive weight of each three-dimensional point. Finally, the data containing the point cloud index, point cloud comprehensive weight, original coordinates and other fields is constructed to obtain the feature weight set. The present scheme considers the influence of various factors on the weight by obtaining various weight-related information of each three-dimensional point, which can more reasonably balance the fusion weight of multi-exposure data, thereby improving the accuracy and reliability of data fusion.

[0111] Then, taking the 1ms point cloud as the reference benchmark, the ICP (Iterative Closest Point) algorithm is used to calculate the spatial transformation relationship (including translation and rotation) between the 1ms point cloud and other exposure time point clouds (such as 5ms, 10ms, etc.), and the coordinate positions of other point clouds are adjusted according to these transformation relationships, so that all point clouds correspond accurately in three-dimensional space, to eliminate the positional deviation between different exposure point clouds caused by possible minor displacements (such as slight device shaking, sample minor movement, etc.) during the scanning process. At the same time, the accuracy of the registration is clearly required, that is, after registration, the positional error of the corresponding points between different point clouds must be less than or equal to 5 microns (microns), to meet the high precision standard of dental scanning. After completing the point cloud alignment, for each target point in the three-dimensional space, the coordinates of all points corresponding to the target point in different exposure time (such as 1ms, 5ms, 10ms, etc.) point clouds need to be found, and the respective comprehensive weights of these corresponding target points in different exposure time point clouds are also obtained. Then, the final fused coordinates of the three-dimensional point are calculated by weighted averaging, specifically: multiply the coordinates of each target point by its own weight, sum them up, and then divide by the sum of all weights (to avoid division by zero error, an extremely small value will be added), to obtain the integrated feature coordinates. This processing method follows the logic that: in the high light area (where overexposure is likely to occur), the point cloud data weight of short exposure time (such as 1ms) is higher, because short exposure can reduce the error caused by overexposure; while in the groove bottom (an area with complex structure and dark light), the point cloud data weight of long exposure time (such as 10ms) is higher, because long exposure can capture more details, combined with curvature weight adjustment, to reduce distortion in this area. Through this targeted data fusion method, the integrated feature coordinates obtained ultimately can take into account the imaging characteristics of different areas, improving the overall accuracy.

[0112] The normal vector is a vector describing the surface orientation of a three-dimensional point (e.g., perpendicular to the surface of the point), but the high reflectivity of enamel (e.g., the surface of a tooth) can cause the normal vector of the point cloud at different exposure times to be deflected (e.g., long exposure may cause the normal vector to be calculated to be deflected in the wrong direction due to reflection), in order to correct this deviation, an antisymmetric matrix related to the enamel reflection parameters (a third-order special matrix describing the law of normal vector deflection caused by the reflection characteristics of enamel, used to describe the mathematical properties of rotation) needs to be constructed first, and then based on the matrix and the exposure time, the rotation compensation matrix is calculated by matrix exponential operation, which quantifies the physical law that "the longer the exposure time, the greater the angle of normal vector deflection caused by reflection", for example, the normal vector of 10ms exposure may need more rotation compensation than 1ms exposure (e.g., a deflection of 11.5° requires a specific rotation compensation matrix), then for each three-dimensional point, its corresponding original normal vector is extracted from the point cloud at different exposure times, and each original normal vector is multiplied by the corresponding rotation compensation matrix to obtain the corrected normal vector of each three-dimensional point, to complete the correction of the reflection deflection, then refer to the logic of coordinate fusion, weight the corrected normal vector with the comprehensive weight at each exposure time, and finally normalize the result to a vector with a length of 1 to ensure the mathematical norm of the normal vector, thereby obtaining the final normal vector of the three-dimensional point, the whole process solves the distortion problem of the normal vector caused by reflection at different exposures through "physical compensation + weighted fusion", makes the final normal vector more accurately reflect the real orientation of the enamel surface, provides a guarantee for the accuracy of the subsequent three-dimensional model, and finally obtains three-dimensional fusion feature data, which includes three core parts: first, the final fusion coordinates after fusion processing; second, the final normal vector after compensation correction; third, the dominant exposure source identifier for marking which exposure time point cloud data (e.g., 1ms, 5ms, etc.) the coordinate mainly depends on, if it is found later that the fusion result of a certain area is biased, the dominant exposure source can be quickly located through the identifier to analyze whether there is a problem with the data collected at that exposure time, which not only embodies the comprehensiveness of fusion, but also retains the ability to trace the core contribution source, and these data are output in the form of PLY format files, the prior art cannot effectively correct the normal vector deflection problem caused by the high reflectivity of enamel, which makes the normal vector unable to accurately reflect the real orientation of the enamel surface, affecting the accuracy of the three-dimensional model, the present scheme can accurately correct the normal vector through physical compensation and weighted fusion, providing a guarantee for the high-precision construction of the three-dimensional model.

[0113] In one embodiment, the first optimization module comprises:

[0114] A distance calculation unit is configured to extract the Gaussian curvature in the three-dimensional fusion feature data, and calculate the geodesic distance from each three-dimensional point to the nearest edge point according to the Gaussian curvature to obtain a geodesic distance set.

[0115] a scattering compensation unit configured to obtain a preset scattering compensation coefficient set and obtain an enamel scattering compensation coordinate of each three-dimensional point according to the scattering compensation coefficient set and the geodesic distance set;

[0116] a neighborhood analysis unit configured to obtain a plurality of three-dimensional point radius neighborhoods and corresponding neighborhood normal vector gradients according to the enamel scattering compensation coordinates, and obtain a shape index and a shape index gradient corresponding to each of the three-dimensional point radius neighborhoods according to the neighborhood normal vector gradients;

[0117] a pit screening unit configured to screen the three-dimensional point radius neighborhoods according to the shape index to obtain a deep pit region, and obtain a conduction coefficient corresponding to the deep pit region according to the shape index gradient;

[0118] an initial optimization unit configured to adjust anisotropic diffusion filtering by the conduction coefficient to obtain pit region point cloud data, and construct an initial optimization data set according to the pit region point cloud data and the enamel scattering compensation coordinates.

[0119] According to the functions of the above units, the present application extracts the Gaussian curvature in the three-dimensional fusion feature data, screens a region with a Gaussian curvature greater than 0.85 (a high curvature region usually corresponds to the edge and corner structure of the occlusal surface), marks the region as the edge of the occlusal surface, and forms a binary edge mask mark (similar to a "marker map", the edge region is 1 and the non-edge region is 0), then takes the three-dimensional point in the point cloud as a node, connects 8 nearest neighbor points (8-neighborhood) around each point, constructs a geodesic map describing the surface connection relationship between points, and simulates the path along the surface, then starts from the edge point set of the edge mask mark, calculates the shortest surface distance from each three-dimensional point to the nearest edge point through parallel calculation to obtain a geodesic distance set, then obtains a preset scattering compensation coefficient set, wherein the scattering compensation coefficient set is a parameter set preset by the system for calculating the enamel scattering depth compensation amount, including an enamel scattering basic coefficient, a brightness sensitivity coefficient, a distance attenuation coefficient, etc., and the enamel scattering depth compensation amount is calculated by the formula "Δz = β * B γ *e -μ*d"Calculate the luminance-depth compensation for each 3D point, where Δz represents the luminance-depth compensation for that 3D point, β represents the enamel scattering baseline coefficient, a system preset parameter (usually 0.15mm) used to determine the reference amplitude of the compensation, B represents the luminance value of that 3D point, γ represents the luminance sensitivity coefficient, a system preset parameter (usually 2.3) used to adjust the degree of influence of luminance on scattering compensation, μ represents the distance attenuation coefficient, a system preset parameter used to control the attenuation rate of the compensation with distance, d represents the geodesic distance of that 3D point, and e represents the natural constant used to express the exponential decay relationship. This formula is a mathematical model built based on the physical optical properties of enamel (hard tissue on the surface of teeth). The core logic is to dynamically calculate the depth deviation compensation caused by scattering based on the luminance and position characteristics of the area. Specifically, the translucent nature of enamel causes light to scatter internally, and the scattering is more severe in high-brightness areas (luminance values ​​close to 1, such as the center of the pit), and the measured depth will be shallower than the actual depth ("appears shallower"). Therefore, by using "B..." γ "Enhancement compensation, the higher the brightness, the better." γ The larger the value, the greater the luminance-depth compensation and the stronger the correction. The scattering effect is more significant in the edge region of the occlusal surface, while the scattering is weaker in the flat region far from the edge. Therefore, by using "e"... -μ*d "The exponential decay of the compensation amount increases with distance from the edge (greater geodetic distance)," e -μ*d "The smaller the value, the smaller the brightness-depth compensation, to avoid overcompensating areas with weak scattering influence. The enamel scattering baseline coefficient serves as a benchmark to control the overall compensation scale. The brightness sensitivity coefficient and distance attenuation coefficient adjust the sensitivity of brightness and distance to compensation, respectively. The combination of these three factors ensures that the compensation amount accurately matches the physical laws of enamel scattering, ultimately used to correct the depth error of the three-dimensional coordinates. Finally, coordinate correction is required for each three-dimensional point in the 3D fusion feature data. The direction and magnitude of the correction follow two key rules: first, the correction direction is adjusted along the opposite direction of the final normal vector of the three-dimensional point, and the final normal vector is a vector perpendicular to the surface where the point is located (referring to..."). The direction of the glaze scattering is towards the outer side of the surface, while the opposite direction is towards the inner side of the surface (or the direction of the depression). This is because glaze scattering causes the original measurement value to be "shallow" (i.e., the position of the point is closer to the outer side of the surface than it actually is). Therefore, it is necessary to correct it inward to match the true depth. Secondly, the correction range is determined by the brightness-depth compensation amount calculated above, so as to obtain the glaze scattering compensation coordinates of each three-dimensional point. By dynamically calculating the compensation amount based on the physical optical properties of glaze and correcting the depth error, the three-dimensional coordinates can be made closer to the real anatomical structure. This method can solve the problem that the existing technology cannot effectively consider the influence of glaze scattering on depth measurement, resulting in a large depth error in the three-dimensional coordinates.

[0120] Since the prior art cannot maintain edge sharpness while optimizing internal geometric precision when processing the point cloud data of the pit and groove area, and is prone to cause local geometric distortion due to noise or excessive processing, the scheme is aimed at each three-dimensional point in the point cloud, and the three-dimensional point radius neighborhood is determined with the point as the center and 0.2mm as the radius, and the neighborhood normal vector gradient in the three-dimensional point radius neighborhood is calculated by using PCA (principal component analysis), wherein the neighborhood normal vector gradient refers to the spatial change rate of the surface normal vector in the three-dimensional point radius neighborhood, and then two key parameters describing the surface curvature degree, i.e., the maximum principal curvature and the minimum principal curvature, are obtained by fitting a quadratic surface (simulating the bending form of the local surface), then the maximum principal curvature and the minimum principal curvature are substituted into the shape index calculation formula to obtain the shape index of the region, and the value range of the index is [-1, 1], which can quantitatively distinguish the concave-convex form of the region: when the shape index is close to 1, it represents that the region corresponds to a spherical concave surface (such as the deep pit and groove of teeth, which is a feature that needs to be paid attention to); when the shape index is close to -1, it represents that the region corresponds to a spherical convex surface (such as a tooth tip, which belongs to a convex structure); when the shape index is close to 0, it represents that the region corresponds to a plane or a saddle-shaped transition area, and a threshold value of 0.85 is set, the shape index corresponding to each region is compared with the threshold value, if the shape index is greater than the threshold value, the region is determined as a deep pit and groove region, and a binary mask (a marking tool, wherein the pit and groove region is marked as 1, and the non-pit and groove region is marked as 0) is generated, then the shape index gradient of each region is calculated according to the shape index, and the shape index gradient reflects the change rate of the shape index in space, wherein the shape index of the pit and groove interior changes gently, and the shape index gradient is small, while the shape index of the pit and groove edge rapidly transitions from the high value in the pit and groove to the low value outside, and the shape index gradient is large, and then the conduction coefficient is calculated: the square of the shape index gradient is first calculated, and then the square value is divided by the scale parameter (fixed as 0.3mm) of the square, to obtain a ratio, then take the opposite of the ratio as the index of the natural constant, calculate the index power of the natural constant, and the result is the conduction coefficient. Specifically, when the shape index gradient is small, the ratio of the square of the shape index gradient to the square of the scale parameter is also small, the opposite of the ratio is close to 0, the 0 power of the natural constant is close to 1, and therefore the conduction coefficient is close to 1. When the shape index gradient is large, the corresponding ratio is large, the opposite of the ratio is a negative number with a large absolute value, and the negative power of the natural constant is close to 0, so the conduction coefficient is close to 0. Through such a calculation method, the conduction coefficient can be automatically adjusted according to the size of the shape index gradient, realizing the reverse correlation with the shape index gradient. The conduction coefficient is the "switch" and "regulator" of the anisotropic diffusion filtering, directly determining the filtering strength. Finally, only the filter is applied to the groove region (marked by the binary mask). The groove interior is strongly smoothed to eliminate measurement noise and make the groove bottom more regular, and the groove edge is weakly smoothed to retain the clear boundary of the edge and avoid edge blurring. Ultimately, the point cloud data of the groove region after anisotropic diffusion filtering is obtained, thereby optimizing the geometric precision of the groove interior while maintaining the edge sharpness, providing more reliable three-dimensional morphological data for subsequent clinical applications. After completing the anisotropic diffusion filtering, for a three-dimensional point in the marked groove region, the principal curvature direction of the point is first determined, and then a plurality of points adjacent to the current point (usually points close in distance along the direction, such as points with a very small distance) are selected in the principal curvature direction. The principal curvatures of these adjacent points are calculated, and then the principal curvature derivative is obtained by dividing the difference between the principal curvatures of two adjacent points by the actual distance between the two points in the principal curvature direction. When the absolute value of the calculated principal curvature derivative exceeds a threshold value, it indicates that the curvature change in this region is too steep and does not conform to the smooth transition characteristics of the natural morphology of the dental groove. At this time, the coordinates of the point need to be fine-tuned: moving the position of the point along the principal curvature direction, and the distance of each adjustment does not exceed 1 micrometer. Through such fine adjustment, the constrained and optimized point cloud data is obtained to ensure that the curvature change in the groove region is gentle and conforms to the physiological morphology, avoiding local geometric distortion caused by excessive smoothing or noise, and ultimately improving the anatomical rationality of the point cloud data. Based on the above obtained data, the initial optimized data set is finally obtained. The initial optimized data set is a PLY format file containing multiple key attributes. The file integrates the results of all previous processing, and comprehensively considers the geometric precision, surface orientation, optical characteristics, key region identification, and processing trace information, providing a complete and accurate data basis for subsequent clinical applications such as generating DICOM compatible STL models.

[0121] In one embodiment, the second optimization module comprises:

[0122] a hierarchical division unit configured to obtain a vertical distance of each three-dimensional point according to the three-dimensional fusion feature data, and divide each three-dimensional point according to the vertical distance to obtain layer structure information;

[0123] an optical modeling unit configured to obtain a scattering coefficient set according to the layer structure information and the vertical distance, and obtain a structured optical parameter field according to the scattering coefficient set, the layer structure information and the initial optimization data set;

[0124] a photon distribution unit configured to obtain a region division type according to the feature parameter and the structured optical parameter field, and obtain a photon emission angle and a photon distribution number according to the region division type;

[0125] a path calculation unit configured to obtain a photon motion path according to the photon emission angle and the photon distribution number, and obtain a deviation compensation amount according to the photon motion path;

[0126] a final optimization unit configured to obtain compensation constraint data according to the initial optimization data set, and obtain a final optimization data set according to the compensation constraint data, the deviation compensation amount and the structured optical parameter field.

[0127] As the function of each unit described above, the present application obtains stratified structure information by stratified structure division based on the initial optimization data set, wherein the stratified structure information refers to the classification information of the tooth point cloud divided into different layers according to the vertical distance of the three-dimensional point to the enamel surface and the edge mask mark. Specifically, the vertical distance of each three-dimensional point to the enamel surface is calculated based on the final normal vector of the three-dimensional point, and the three-dimensional point is divided into three layers according to the vertical distance and the edge mask mark, and the corresponding refractive index is given. If the three-dimensional point is marked by the edge mask (belongs to the enamel-dentin junction), it belongs to the dentin layer, and the refractive index is fixed at 1.55; if the vertical distance exceeds 0.2 mm, it belongs to the transition layer, and the refractive index linearly changes from 1.63 of the enamel surface layer to 1.55 of the dentin layer; if the vertical distance does not exceed 0.2 mm, it belongs to the enamel surface layer, and the refractive index is fixed at 1.63; for the three layers divided above, the scattering coefficient describing the light scattering ability is calculated, and then the scattering coefficient set is obtained according to the stratified structure information and the vertical distance, wherein the scattering coefficient set refers to a set of scattering coefficients corresponding to different layers. The scattering coefficient of the enamel surface layer decays exponentially with the depth of the vertical distance (first obtain the inverse of the ratio of the vertical distance to 0.2, then take the natural exponential of the result, multiply the result by 0.5, and finally add 0.1, which is the scattering coefficient of the enamel surface layer), to reflect that the scattering coefficient decays exponentially with the increase of the vertical distance, and the surface scattering is strong and gradually weakens with the increase of the vertical distance; the scattering coefficient of the transition layer also decays exponentially with the vertical distance (first obtain the difference between the vertical distance and 0.2, take the inverse of the ratio of the difference to 0.15, then take the natural exponential of the result, multiply the result by 0.3, and finally add 0.05, which is the scattering coefficient of the transition layer), which continues the decay trend, but the decay rate and the basic value are different from those of the enamel layer; the scattering coefficient of the dentin layer is fixed at 0.4 (to simulate the relatively stable scattering characteristics of dentin), and through the above operation, the structured optical parameter field is finally obtained, which specifically represents a data set containing the detailed optical properties of each point in the three-dimensional point cloud. The data set integrates the spatial coordinates of each three-dimensional point, the belonging layer, the refractive index of the corresponding layer, the scattering coefficient and the penetration depth, etc., providing accurate physical basis for subsequent optical simulation such as photon tracing. The present scheme provides accurate physical basis for optical simulation such as photon tracing through structured division and parameter calculation, so that the subsequent simulation is more consistent with the actual optical characteristics of teeth.

[0128] Then according to the absolute value of the curvature of each three-dimensional point, the region type is divided: when the absolute value of the curvature is greater than 0.1, the point belongs to the "high curvature region" (such as the groove, tooth tip and other parts of the tooth with dramatic morphological changes), at this time the narrow cone angle Gaussian beam with the emission angle equal to 5° is used to emit photons, this narrow angle beam can concentrate the photons in a specific direction, thereby improving the photon density in the shadow area such as the bottom of the groove, and making up for the problem of insufficient photon irradiation caused by the morphological depression; when the absolute value of the curvature is less than or equal to 0.1, the point belongs to the "low curvature region" (such as the flat enamel surface of the tooth and other parts with gentle morphology), at this time the wide cone angle Gaussian beam with the emission angle equal to 15° is used to emit photons, this wide angle beam can cover the flat surface more widely, and enhance the sampling of the scattering condition of these areas, to ensure that the optical characteristics of the gentle region are fully captured. Through this way of dynamically adjusting the emission angle according to the curvature, targeted photon irradiation of different morphological regions is realized, which not only solves the shadow compensation problem of high curvature region, but also ensures the sampling integrity of low curvature region. Then, the cube of the brightness value corresponding to each three-dimensional point is calculated respectively, and the sum of the cube of 2 times the brightness value and 1 is calculated. The product of the obtained sum and the system preset basic photon number is the photon allocation number of the three-dimensional point. Through this way, the photon allocation number increases non-linearly with the increase of the brightness value, thereby realizing the preferential allocation of photons in high brightness regions. Then, for each three-dimensional point in the point cloud, two angle parameters are randomly generated within the cone angle corresponding to the region to which the point belongs: polar angle (describing the angle between the photon emission direction and the normal vector of the point) and azimuth angle (describing the rotation angle of the photon in the plane perpendicular to the normal vector). Then, the two angle parameters are converted into direction vectors in three-dimensional space through mathematical conversion, to determine the emission direction of each photon. Finally, according to the photon allocation number corresponding to the point calculated above, a corresponding number of virtual photons are emitted towards the virtual dental tissue model along the direction vector generated above. Then, the computer simulates the motion of each photon according to the optical physics laws such as scattering theory and energy conservation law - when the photon collides with the medium molecules, scattering (changing the propagation direction) occurs, part of the energy may be absorbed by the medium (causing photon energy attenuation), if the photon reaches the surface of the tissue, reflection or refraction (exit) may occur, this process continues until the photon energy is exhausted (cannot continue to propagate) or is "caught" by the detector, finally forming the photon motion path. The photon motion path refers to the trajectory "running out" in the computer, which is used to simulate the propagation behavior of real light in the tooth, and is the basic data for subsequent analysis (such as light intensity ratio calculation).

[0129] Then, for each photon motion path of the three-dimensional point, the photon motion paths are grouped according to the space points, that is, all the photon motion paths related to the same three-dimensional point are grouped into a group, and then two energy values are calculated for each three-dimensional point: one is the total energy of all photons actually emitted from the point (that is, the sum of the energy of the photons finally emitted from the vicinity of the point after the physical process of scattering, etc.); and the other is the theoretical light intensity of the point under the ideal non-scattering condition queried from the optical simulation pre-computation library (that is, the theoretical energy value assuming no scattering loss or aggregation in the light propagation process), and the ratio of the actual total energy to the theoretical light intensity is calculated to obtain the light intensity ratio, which reflects the influence of scattering on the signal: when the light intensity ratio is greater than 1, it means that the actual emitted photon energy is greater than the theoretical value, which means that scattering causes the aggregation of photons at the point, resulting in false signal enhancement; when the light intensity ratio is less than 1, the actual emitted energy is less than the theoretical value, which means that scattering causes the loss of photons in the propagation, resulting in signal attenuation, then for each three-dimensional point, first calculate the difference between the light intensity ratio of the point and 1, then multiply this difference by the residence time (with a precision of picoseconds, that is, the time for which the photon stays near the point) of all photons of the point, and then add all the products to obtain the time integration result, finally multiply this integration result by the system calibration coefficient (such as 0.08mm) to obtain the value of the deviation compensation amount required by the point, then perform geometric coordinate correction: according to the calculated deviation compensation amount, adjust the spatial coordinates of the point along the final normal vector direction of the point, when the deviation compensation amount is greater than 0, it means that the coordinates of the point need to be moved along the normal direction to compensate for the "shallow measurement" (that is, the measured position is shallower than the actual position) caused by the loss of photons; when the deviation compensation amount is less than 0, the coordinates are moved along the normal direction to compensate for the "deep measurement" (that is, the measured position is deeper than the actual position) caused by the aggregation of photons, since the prior art cannot effectively irradiate photons for different tooth morphology regions, and cannot accurately correct the spatial positioning error caused by light scattering, therefore, the present scheme can improve the photon irradiation effect in different regions, accurately correct the spatial positioning error caused by light scattering, and make the point cloud coordinates closer to the real tooth anatomical structure through dynamic adjustment of the emission angle and accurate coordinate correction.

[0130] Subsequently, the constraint execution process is performed, the curvature radius constraint: the "deep pit" area marked by the binary mask is obtained from the point cloud, for these pit points, the curvature radius thereof is calculated (the inverse of the absolute value of the curvature value corresponding to the point is the curvature radius), the curvature radius reflects the bending degree of the curve, the smaller the curvature radius, the more severe the bending, then a judgment threshold of 0.3mm is set (the threshold is set based on the lower limit value 0.28mm of the curvature radius of the healthy molar pit, as a safety standard), if the calculated curvature radius is less than 0.3mm, it means that the bending degree of the point exceeds the normal range of the healthy tooth, which may be caused by excessive correction before, at this time, the point needs to be adjusted: retreat along its normal by the corresponding deviation compensation amount, and update the coordinate and deviation compensation amount of the point, the purpose of this step is to avoid the coordinate correction of the pit area exceeding the anatomical range of the healthy tooth through the physiological threshold constraint of the curvature radius, to ensure the physiological reasonableness of the model, and to smooth the compensation constraint: finally, for each three-dimensional point in the point cloud, first find its 8 neighboring points (neighborhood points), calculate the average of the deviation compensation amounts of the 8 neighborhood points, then compare the deviation compensation amount of the point itself with the average, if the difference between the two exceeds 0.02 (the unit is usually mm, representing the set allowed fluctuation threshold), it means that the compensation amount of the point has obvious mutation with the surrounding area, which may cause local unevenness of the subsequent model, at this time, the deviation compensation amount of the point needs to be smoothed: adopt the weighted average method, add 0.7 times the deviation compensation amount of the point itself to 0.3 times the average of the neighborhood points to obtain the deviation correction compensation amount, and finally adjust the coordinate of the point according to the deviation correction compensation amount, the purpose of this step is to eliminate the sharp change of the local compensation amount, so that the coordinate correction of the entire point cloud is more smooth and continuous, thereby avoiding stress concentration caused by local structure mutation of the restoration body when in place (which may cause damage to the restoration body or tooth tissue), and ensuring the adaptability and safety of the restoration body; based on the above, the final optimized data set is obtained, each point in the point cloud adds an attribute mark, wherein "0001" indicates that the point triggers the curvature radius constraint and is corrected, and "0010" indicates that the smoothing constraint is triggered and corrected (the binary bit mark can clearly trace whether the point has been intervened and the specific intervention type); the constraint correction report (stored in JSON format) records the number of cases triggering the curvature constraint and the smoothing constraint, as well as the statistical data of the correction amount of each point (such as the range and average of the correction value), this method avoids the deviation of the model from the physiological characteristics by incorporating the clinical anatomical constraint (such as the curvature radius threshold of the healthy tooth pit), solves the problem of loss of subtle features such as pits caused by global smoothing processing in traditional technology by using local adaptive constraint (only the abnormal points exceeding the threshold are corrected, and the normal points are not intervened), and realizes the traceability of the compensation amount by recording the constraint intervention, which makes up for the defect that the traditional method cannot trace the correction process.

[0131] In one embodiment, the model construction module comprises:

[0132] a model initial construction unit, configured to acquire simplified point cloud data, curvature enhancement weight and core configuration parameters according to the final optimization data set, and acquire a preliminary three-dimensional model according to the core configuration parameters, the curvature enhancement weight and the simplified point cloud data;

[0133] a model processing unit, configured to acquire a plurality of vertex radius neighborhoods and corresponding vertex densities according to the preliminary three-dimensional model, and screen a plurality of the vertex radius neighborhoods according to the vertex densities to obtain a topological optimization model;

[0134] a groove depth acquisition unit, configured to acquire a groove region depth according to the topological optimization model;

[0135] a parameter optimization unit, configured to acquire a groove basic depth, acquire a groove depth error according to the groove basic depth and the groove region depth, and adjust the core configuration parameters according to the groove depth error to obtain optimized configuration parameters;

[0136] a final die generation unit, configured to acquire a tooth three-dimensional model according to the optimized configuration parameters, the curvature enhancement weight and the simplified point cloud data.

[0137] As the function of each unit described above, the application obtains the simplified point cloud data by downsampling processing based on the final optimization data set according to the voxel size of 20 microns. Voxelization is to divide the continuous point cloud space into discrete cubic small grids (voxels), and only one representative point is retained in each voxel, thereby reducing the number of points and reducing the subsequent calculation complexity on the premise of ensuring accuracy. Then, curvature weight enhancement is implemented. The points in the groove region are identified through the original color markers in the point cloud (the color markers are the distinguishing marks previously set for the points in the groove region), and the points in the groove region are given a weight of 1.3 times that of ordinary points (i.e. the weight is increased by 30%), to obtain the curvature enhancement weight. The purpose is to enhance the preservation of important anatomical features such as grooves in the subsequent three-dimensional reconstruction process, and to avoid the loss or blurring of features due to the global smoothing effect of the algorithm. Then, the core configuration parameters are obtained according to the final optimization data set, wherein the core configuration parameters refer to a set of key parameters used to control the three-dimensional reconstruction algorithm to meet the needs of tooth three-dimensional model reconstruction: the octree depth is set to 10. This setting can provide a resolution that meets the micron-level accuracy requirements of dentistry by recursively dividing the three-dimensional space into 1024 levels of grid while ensuring computational efficiency; the conjugate gradient method is selected as the linear solver. This method is suitable for handling large-scale data and has better stability for the "ill-conditioned matrix" that may occur in the tooth model due to uneven data distribution, and can effectively solve the complex equation set in the reconstruction process; at the same time, the weight data of the groove region points increased by 30% before is injected into the algorithm, so that the details of the high-weight region can be preferentially preserved during reconstruction, avoiding the smoothing processing of important features such as grooves; the scale parameter is set to 1.1, which slightly enlarges the reconstruction result to compensate for the possible "model shrinkage" problem of the algorithm, ensuring that the final model size is closer to the real tooth; in addition, 8-thread GPU acceleration is enabled to greatly improve the operation speed of the reconstruction process and shorten the overall processing time. These parameter configurations are specially optimized for the anatomical features of the tooth model and the clinical accuracy requirements, providing a strong algorithmic guarantee for generating high-quality three-dimensional models in the subsequent process. Then, through octree division of the space, the conjugate gradient method is used to solve the Poisson equation to fit the implicit surface of the point cloud distribution, and the groove details are preferentially preserved by combining the curvature weight, to finally generate a preliminary three-dimensional model containing information such as triangular facets and vertex coordinates. This method of reducing calculation complexity through downsampling, preserving important features such as grooves through curvature weight enhancement, and improving model generation quality and efficiency through optimized parameter configuration can effectively solve the problem of easy loss of important anatomical features in the generation of three-dimensional models due to high calculation complexity in the prior art.

[0138] Then the preliminary three-dimensional model needs to be optimized to improve the quality. After generating the preliminary model by Poisson reconstruction, the algorithm traverses all the triangular facets of the model, constructs a vertex radius neighborhood for each vertex (usually defined by a fixed radius or K adjacent vertices), and for each vertex, counts the number of other vertices contained in its neighborhood range, and combines the surface area of the neighborhood (calculated by summing the areas of the triangular facets) to obtain the number of vertices per unit area, i.e. the vertex density of the region. This density value indirectly reflects the data distribution characteristics of the original point cloud: areas with dense original point clouds (such as rich details such as grooves) have higher vertex density after reconstruction, while areas with sparse point clouds or noise have lower vertex density. Based on the calculated vertex density, the part with density higher than the overall 10% quantile is retained, and the area with too low density is removed. These low-density areas are mostly noise or redundant structures, and removing them can reduce the invalid details of the model and make the topology more concise. Then, Laplace smoothing is performed by iterating 3 times with a smoothing coefficient of 0.5 to fine-tune the model surface. This operation can weaken the small protrusions or depressions caused by point cloud noise (suppress noise), while the moderate smoothing coefficient (not biased towards excessive smoothing) can preserve important sharp edge features such as groove edges and tooth tips (avoid feature blurring). After the above optimization, the processed three-dimensional model is output in STL format to obtain the topology-optimized model.

[0139] Subsequently, while generating the final three-dimensional model, the accuracy of the key anatomical feature (groove) needs to be verified: the algorithm measures the depth of the groove region in the reconstructed model and compares it with the basic depth of the groove, calculates the difference between the two, and obtains the groove depth error. If the groove depth error exceeds the preset threshold of 35 microns (the upper limit of clinically acceptable accuracy), the system will automatically adjust the core parameters of Poisson reconstruction to obtain optimized configuration parameters (such as increasing the octree depth from the original 10 to 11). An increase in octree depth means an increase in the level of spatial division (from 2^10 to 2^11), resulting in a higher grid resolution that can capture more subtle structural features, thereby optimizing the resolution of the model. This dynamic adjustment mechanism ensures that when the preliminary reconstruction result does not meet the accuracy requirements, the algorithm resolution is increased to recalculate, ultimately controlling the groove depth error within 35 microns to ensure the clinical applicability of the model. After the above series of processing procedures, a tooth three-dimensional model (STL format) that meets the specification standards is finally generated. This optimization process removes noise and redundant structures, improves the surface quality of the model, and ensures the clinical applicability of the model through accuracy verification and parameter adjustment.

[0140] The application also provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the memory is configured to execute the computer program to manage the operation of the computer vision-based tooth three-dimensional modeling system.

[0141] The application further provides a computer readable storage medium, which stores a computer program, and the computer program is used for managing the operation of the computer vision-based tooth three-dimensional modeling system when executed by a processor.

[0142] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware, and the computer program can be stored in a non-volatile computer readable storage medium, and when executed, the computer program can include the processes of the above-mentioned embodiments. Any reference to memory, storage, database or other medium provided by the present application and used in the embodiments can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM) and memory bus dynamic RAM (RDRAM) and the like.

[0143] It should be noted that in this document, the terms "comprising", "containing", or any other similar term are intended to encompass non-exclusive inclusion, so that a process, device, article or method including a list of elements not only includes those elements, but also includes other elements not explicitly listed, or inherent to such a process, device, article or method. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, device, article or method including the element.

[0144] The above only describes the preferred embodiments of the present application, and does not limit the patent scope of the present application, and any equivalent structure or equivalent process transformation using the content of the present application specification and drawings, or direct or indirect application in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A computer vision based three-dimensional modeling system for teeth, comprising: The method comprises the following steps: a data acquisition module is configured to acquire tooth multi-modal core data based on computer vision, wherein the multi-modal core data comprises original image data and camera calibration data; a texture generation module is configured to acquire a two-dimensional texture image based on the original image data; a point cloud reconstruction module is configured to acquire a projected original image based on the two-dimensional texture image, and acquire a complete point cloud data set based on the projected original image and the camera calibration data; a sequence acquisition module is configured to acquire a plurality of exposure point cloud sequences based on the camera calibration data, the two-dimensional texture image, and the complete point cloud data set; a feature fusion module is configured to fuse the plurality of exposure point cloud sequences to obtain three-dimensional fused feature data; a first optimization module is configured to construct an initial optimization data set based on the three-dimensional fused feature data; a second optimization module is configured to construct a final optimization data set based on the initial optimization data set; a model construction module is configured to construct a tooth three-dimensional model based on the final optimization data set.

2. The computer vision based three-dimensional modeling system of teeth of claim 1, wherein, The texture generation module comprises: a component extraction unit configured to acquire a specular reflection reference image, a diffuse reflection background image, and a standard brightness image based on the original image data, and acquire a specular reflection component, a diffuse reflection component, and a subsurface scattering component based on the specular reflection reference image and the diffuse reflection background image; a coefficient solving unit configured to acquire a target constraint condition, and acquire a specular reflection coefficient, a diffuse reflection coefficient, and a subsurface scattering coefficient based on the specular reflection component, the diffuse reflection component, the subsurface scattering component, and the target constraint condition; a first acquisition unit configured to acquire a specular reflection relative intensity of a plurality of pixel points based on the specular reflection coefficient, the diffuse reflection coefficient, and the subsurface scattering coefficient, and acquire a specular reflection intensity map based on the specular reflection relative intensity of each pixel point; a reflection alignment unit configured to acquire a reflection feature alignment map based on the standard brightness image and the specular reflection intensity map, and acquire a diffuse reflection proportion map based on the reflection feature alignment map; a texture synthesis unit configured to acquire a two-dimensional texture image based on the diffuse reflection proportion map, the specular reflection reference image, and the diffuse reflection background image.

3. The computer vision based three-dimensional modeling system of teeth of claim 1, wherein, The point cloud reconstruction module comprises: a feature extraction unit configured to acquire a gray level histogram and an enamel pit average width based on the two-dimensional texture image, and acquire a high reflection area coordinate based on the gray level histogram; a projection generation unit configured to acquire a fundamental frequency, a second harmonic, and a third harmonic based on the enamel pit average width, and acquire a projected original image based on the fundamental frequency, the second harmonic, and the third harmonic; a projection correction unit configured to acquire a basic compensation amount based on the camera calibration data, and acquire a projection correction image based on the basic compensation amount and the projected original image; an exposure optimization unit configured to acquire an enamel area proportion and a saturated pixel proportion based on the projection correction image, and acquire a multi-exposure sequence image set based on the enamel area proportion; a three-dimensional initial construction unit configured to acquire a three-dimensional point cloud set based on the multi-exposure sequence image set. The point cloud integration unit is configured to acquire an exposure time-depth error correspondence table, and acquire a complete point cloud data set according to the exposure time-depth error correspondence table, a three-dimensional point cloud set and the proportion of saturated pixels.

4. The computer vision based three-dimensional modeling system of teeth of claim 1, wherein, The sequence acquisition module comprises: The parameter calibration unit is configured to acquire intrinsic parameters of the calibration board, initial calibration parameters and an initial extrinsic parameter matrix according to the camera calibration data, and acquire three-dimensional coordinates of feature points according to the intrinsic parameters of the calibration board; The feature detection unit is configured to acquire two-dimensional coordinates of feature points and effective pixel brightness according to the two-dimensional texture image; The bias acquisition unit is configured to acquire theoretical projection coordinates according to the intrinsic parameters of the calibration board and the initial calibration parameters, and acquire pixel coordinate bias according to the theoretical projection coordinates and the two-dimensional coordinates of the feature points; The extrinsic parameter optimization unit is configured to acquire an optimized extrinsic parameter matrix according to the initial extrinsic parameter matrix and the pixel coordinate bias, and acquire normal vectors of feature points according to the optimized extrinsic parameter matrix and the three-dimensional coordinates of the feature points; The sequence generation unit is configured to acquire a brightness-normal vector mapping rule according to the effective pixel brightness and the normal vectors of the feature points, and acquire a multi-exposure point cloud sequence according to the complete point cloud data set, the brightness-normal vector mapping rule and the optimized extrinsic parameter matrix.

5. The computer vision based three-dimensional modeling system of teeth of claim 1, wherein, The feature fusion module comprises: The parameter extraction unit is configured to extract feature parameters of each three-dimensional point in a plurality of exposure point cloud sequences, wherein the feature parameters comprise exposure time, brightness value, gradient amplitude, Gaussian curvature and reflectance probability; The weight construction unit is configured to assign weights to the feature parameters respectively to obtain a feature weight set containing weight information; The coordinate fusion unit is configured to acquire a preset registered point cloud sequence, and acquire final fusion coordinates of each three-dimensional point according to the registered point cloud sequence, the plurality of exposure point cloud sequences and the feature weight set; The normal vector correction unit is configured to acquire an anti-symmetric matrix and an original normal vector of each three-dimensional point, acquire a rotation compensation matrix according to the anti-symmetric matrix and the exposure time, and correct the original normal vector according to the rotation compensation matrix to obtain a final normal vector; The feature integration unit is configured to fuse the final fusion coordinates and the final normal vector to obtain three-dimensional fused feature data.

6. The computer vision based three-dimensional modeling system of teeth of claim 1, wherein, The first optimization module comprises: The distance calculation unit is configured to extract Gaussian curvature in the three-dimensional fused feature data, and calculate geodesic distances from each three-dimensional point to the nearest edge point according to the Gaussian curvature to obtain a geodesic distance set; The scattering compensation unit is configured to acquire a preset scattering compensation coefficient set, and acquire enamel scattering compensation coordinates of each three-dimensional point according to the scattering compensation coefficient set and the geodesic distance set; The neighborhood analysis unit is configured to acquire a plurality of three-dimensional point radius neighborhoods and corresponding neighborhood normal vector gradients according to the enamel scattering compensation coordinates, and acquire a shape index and a shape index gradient corresponding to each three-dimensional point radius neighborhood according to the neighborhood normal vector gradients; The pit and groove screening unit is configured to screen the three-dimensional point radius neighborhoods according to the shape index to obtain a deep pit and groove region, and acquire a conduction coefficient corresponding to the deep pit and groove region according to the shape index gradient; An initial optimization unit is configured to perform anisotropic diffusion filtering by adjusting the conduction coefficient to obtain a pit and fissure region point cloud data, and to construct an initial optimization data set according to the pit and fissure region point cloud data and the enamel scattering compensation coordinates.

7. The computer vision based three-dimensional modeling system of teeth of claim 5, wherein, The second optimization module comprises: A hierarchical division unit is configured to obtain a vertical distance of each three-dimensional point according to the three-dimensional fusion feature data, and to divide each three-dimensional point according to the vertical distance to obtain layer structure information; An optical modeling unit is configured to obtain a scattering coefficient set according to the layer structure information and the vertical distance, and to obtain a structured optical parameter field according to the scattering coefficient set, the layer structure information and the initial optimization data set; A photon distribution unit is configured to obtain a region division type according to the feature parameters and the structured optical parameter field, and to obtain a photon emission angle and a photon distribution number according to the region division type; A path calculation unit is configured to obtain a photon motion path according to the photon emission angle and the photon distribution number, and to obtain a deviation compensation amount according to the photon motion path; A final optimization unit is configured to obtain compensation constraint data according to the initial optimization data set, and to obtain a final optimization data set according to the compensation constraint data, the deviation compensation amount and the structured optical parameter field.

8. The computer vision based three-dimensional modeling system of teeth of claim 1, wherein, The model construction module comprises: A model initial construction unit is configured to obtain a simplified point cloud data, a curvature enhancement weight and a core configuration parameter according to the final optimization data set, and to obtain a preliminary three-dimensional model according to the core configuration parameter, the curvature enhancement weight and the simplified point cloud data; A model processing unit is configured to obtain a plurality of vertex radius neighborhoods and corresponding vertex densities according to the preliminary three-dimensional model, and to screen a plurality of the vertex radius neighborhoods according to the vertex densities to obtain a topological optimization model; A groove depth obtaining unit is configured to obtain a pit and fissure region depth according to the topological optimization model; A parameter optimization unit is configured to obtain a pit and fissure basic depth, to obtain a pit and fissure depth error according to the pit and fissure basic depth and the pit and fissure region depth, and to adjust the core configuration parameter according to the pit and fissure depth error to obtain an optimized configuration parameter; A final model generation unit is configured to obtain a tooth three-dimensional model according to the optimized configuration parameter, the curvature enhancement weight and the simplified point cloud data. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. The memory is configured to manage the operation of the computer vision-based tooth three-dimensional modeling system according to any one of claims 1 to 8 when the computer program is executed.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is configured to manage the operation of the computer vision-based tooth three-dimensional modeling system according to any one of claims 1 to 8 when executed by the processor.

Images