Oral cavity three-dimensional model texture mapping method and device for soft tissue

Abstract

The invention provides an oral cavity three-dimensional model texture mapping method and device for soft tissue. The method comprises the following steps: scanning and collecting a two-dimensional image and three-dimensional point cloud data of an oral cavity; carrying out panoramic fusion and soft and hard tissue segmentation processing on the two-dimensional image; adopting an off-line parameterization method for the separated soft tissue part, and reconstructing a soft tissue three-dimensional model by setting deformation parameters and based on iterative optimization of segmented fusion; and mapping the texture information of the two-dimensional image to the three-dimensional model, and optimizing and screening the texture information through a probability graph to complete the three-dimensional model texture mapping of the oral soft tissue. Through a differentiated and efficient data processing method, rapid and accurate oral cavity three-dimensional model reconstruction and texture mapping are realized.

Description

Technical Field

[0001] The present invention relates to the field of oral three - dimensional model reconstruction and texture mapping, and particularly to a texture mapping scheme for oral three - dimensional models of soft tissues. Background Art

[0002] With the rapid development of technology, the construction of three - dimensional models and texture mapping technology are increasingly widely used in the field of oral medicine, playing an important role in orthodontics, restoration, and the diagnosis and treatment planning of oral diseases. Traditional methods for constructing oral three - dimensional models mainly include plaster molding and two - dimensional imaging techniques, but these methods have significant limitations. For example, the method based on manual molding is not only inefficient and limited in precision, but also difficult to handle complex oral structures and cannot intuitively display details such as the color and texture of teeth and soft tissues; while the method based on two - dimensional imaging often results in information loss due to perspective limitations and projection distortion when reconstructing three - dimensional structures, making it difficult to accurately reflect the complex internal structure of the oral cavity. These limitations have to some extent affected the accuracy of clinical diagnosis and the formulation of treatment plans.

[0003] In recent years, with the rapid development of oral three - dimensional scanning technology, handheld oral three - dimensional scanners have gradually become popular, capable of simultaneously collecting two - dimensional RGB images and three - dimensional point cloud data. However, how to efficiently and accurately map the texture information of two - dimensional images onto three - dimensional point cloud models is still a technical problem to be solved urgently. Especially in the three - dimensional reconstruction and texture mapping of oral soft tissues, existing technologies often have difficulty achieving precise and rapid texture mapping and model reconstruction. Due to the complex geometric shape and non - rigid characteristics of soft tissues, existing texture mapping methods cannot effectively maintain the continuity and accuracy of their textures, which further increases the difficulty of technical implementation.

[0004] Therefore, developing an efficient and accurate texture mapping method for oral three - dimensional models of soft tissues, especially rapid texture mapping for oral soft tissues, is of great significance for improving the accuracy of oral medical diagnosis and treatment. This method needs to overcome the challenges brought by non - rigid deformation of soft tissues to achieve high - quality three - dimensional model reconstruction and texture mapping. Summary of the Invention

[0005] The present invention provides a texture mapping method for oral three - dimensional models of soft tissues. This method is based on a handheld oral three - dimensional scanner to collect two - dimensional images and three - dimensional point clouds. After panoramic image fusion of the two - dimensional images, hard and soft tissue separation processing is carried out. Then, an offline parameterization method is adopted for the separated non - rigid soft tissues, deformation parameters are set, and through iterative optimization of piece - by - piece fusion, the deformed voxels are used to accurately describe the tiny deformation of soft tissues, thereby accurately reconstructing the soft tissue model, and finally high - definition texture mapping is performed.

[0006] To achieve the above object, a method for texture mapping of oral three-dimensional models for soft tissues designed by the present invention is characterized in that for the non-rigid deformation of the soft tissue region, an offline parameterized three-dimensional reconstruction method is adopted after separating it from the hard tissue, which not only ensures the accuracy of the three-dimensional reconstruction and texture mapping of the soft tissue part, but also improves the efficiency of the overall oral cavity modeling.

[0007] To solve the above technical problems, the technical solution of the present invention is a method for texture mapping of oral three-dimensional models for soft tissues, including the following processes: Scan and collect two-dimensional images and three-dimensional point cloud data of the oral cavity; Perform panoramic fusion and hard and soft tissue segmentation processing on the two-dimensional images; For the separated soft tissue part, adopt an offline parameterization method, and reconstruct the three-dimensional model of the soft tissue by setting deformation parameters and based on iterative optimization of segmented fusion; Map the texture information of the two-dimensional image to the three-dimensional model, and optimize and screen the texture information through a probability map to complete the texture mapping of the three-dimensional model of the oral soft tissue.

[0008] Moreover, during acquisition, noise filtering and redundant frame removal are performed on the three-dimensional point cloud data, and the redundant frame removal is based on the overlap rate between adjacent frames.

[0009] Moreover, the hard and soft tissue segmentation adopts an improved U-Net model. An attention mechanism is added between the encoder and the decoder of the U-Net model, and the boundary region between the hard and soft tissues is enhanced through an attention gate, and a segmentation mask is output.

[0010] Moreover, for the feature point extraction of the soft tissue, a deformable part model is adopted, for the feature point extraction of the hard tissue, the ORB algorithm is adopted, and the two-dimensional image features and the three-dimensional point cloud features are matched through a cross-modal Transformer model.

[0011] Moreover, when performing offline parameterized three-dimensional reconstruction, the soft tissue point cloud is converted into a voxel grid, translation, rotation and scaling deformation parameters are defined for each voxel, and the deformation parameters are solved by optimizing the objective function.

[0012] Moreover, optimization of the voxel grid is performed, including dividing the soft tissue into several small regions based on the K-Means algorithm, and each region is regarded as an approximate rigid body for segmented fusion iterative optimization until all voxels are fused into an overall.

[0013] Moreover, the texture mapping includes calculating the color variance of the texture region corresponding to the triangular patch, generating a probability map and retaining the texture regions with probability values higher than the corresponding threshold, and performing local and global color adjustment processing.

[0014] On the other hand, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the above-mentioned method for texture mapping of the three-dimensional oral model for soft tissues is implemented.

[0015] On the other hand, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned method for texture mapping of the three-dimensional oral model for soft tissues is implemented.

[0016] On the other hand, the present invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, the above-mentioned method for texture mapping of the three-dimensional oral model for soft tissues is implemented.

[0017] The present invention has the following positive effects: 1) The present invention proposes a method for texture mapping of the three-dimensional oral model for soft tissues. Through a differential and efficient data processing method, rapid and accurate reconstruction and texture mapping of the three-dimensional oral model are achieved.

[0018] 2) The present invention realizes differential adaptation processing for the different rigidity characteristics by pre-segmenting hard and soft tissues in panoramic two-dimensional images. For example, different methods are adopted for feature point extraction, three-dimensional reconstruction, etc. This differential adaptation processing ensures the modeling accuracy and improves the calculation efficiency.

[0019] 3) In view of the non-rigid deformation characteristics of soft tissues, the present invention adopts an offline parametric three-dimensional reconstruction method, significantly improving the reconstruction accuracy of soft tissues in the oral cavity.

[0020] 4) The present invention adopts probability map optimization to screen texture information, ensuring high-quality texture mapping effects.

[0021] The present invention provides an efficient solution to the technical difficulties in soft tissue reconstruction during the three-dimensional oral modeling process. This method has broad application potential in many fields such as dental diagnosis and treatment planning, oral medicine digitization, telemedicine, and dental research and development. It can significantly improve the accuracy and efficiency of dental digitization and promote the further development of dental technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to make the technical solutions of the present invention clearer, the drawings included in the document will be briefly described below. It should be clear that these drawings only represent several embodiments of the present invention. For those skilled in the relevant art, without additional creative work, they can fully derive other possible drawings based on these drawings.

[0023] Figure 1A texture mapping process for an oral three-dimensional model of soft tissue according to an embodiment of the present invention.

[0024] Figure 2 The network structure diagram of the improved U-Net model according to an embodiment of the present invention.

[0025] Figure 3 The specific network structure diagram of the improved U-Net model according to an embodiment of the present invention. Detailed implementation manners

[0026] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0027] The present invention proposes a texture mapping method for an oral three-dimensional model of soft tissue, including the following processes: using a handheld oral three-dimensional scanner to obtain two-dimensional images and three-dimensional point cloud data, preprocessing the data to reduce redundant data, performing panoramic fusion on the two-dimensional images, performing hard and soft tissue segmentation based on the two-dimensional panoramic images, extracting feature points from the two-dimensional panoramic images and three-dimensional point clouds, dividing the point clouds based on feature point matching, introducing a parameterized distorted voxel grid method for the soft tissue part of the point clouds, setting an objective function for optimization, performing iterative optimization through regional segmental fusion to make the distorted voxel grid closer to the deformation of the real soft tissue, finally performing texture mapping, and screening the texture based on probability map optimization, and then performing local color adjustment and global color adjustment on the texture information. This method overcomes the challenges brought by the non-rigid deformation of soft tissue, realizes high-quality oral three-dimensional model reconstruction and texture mapping, has important significance for the diagnosis of oral diseases, has wide application value in the fields of dental diagnosis and treatment, telemedicine, and virtual reality education, and promotes the innovative development of dental technology.

[0028] As Figure 1 shown, an embodiment of the present invention proposes a texture mapping method for an oral three-dimensional model of soft tissue, including the following steps: Step 1), oral data collection and preprocessing: The key point of this step is to ensure that the handheld oral three-dimensional scanner can collect high-quality two-dimensional RGB images and three-dimensional point cloud data. By preprocessing the data collected by the handheld oral three-dimensional scanner, useless data can be removed, the data volume can be reduced, which is beneficial to improving the real-time modeling efficiency of oral scanning.

[0029] The preferred implementation method recommended in the embodiment is: Step 1.1, Obtain two-dimensional images and three-dimensional point cloud data: Use a handheld oral three-dimensional scanner with high measurement accuracy (preferably with an accuracy higher than 10 microns) to perform oral scans on all upper and lower teeth and the soft tissues around the teeth, and obtain two-dimensional images and three-dimensional point cloud data inside the oral cavity.

[0030] It is preferably recommended to perform standardized oral scans under standard conditions. The standard environment means that the temperature is 18°C - 30°C, the humidity is 30% - 70%, there is no strong light interference, it is clean and dust-free, the patient's head is stable, and the oral cavity is relatively dry. Standardized oral scans mean scanning in a fixed order. It is preferably recommended that the scanning order be to first scan the outer side (buccal surface) of the teeth, then scan the inner side (lingual surface), and then scan the occlusal surface of the teeth, and ensure that when scanning, the entire surface of the teeth and the required soft tissue parts are covered to avoid missing any details. Specifically, products such as the 3DIFY-JMO1 oral scanner can be selected during implementation.

[0031] Step 1.2, Process the collected three-dimensional point cloud data, such as noise filtering and removal of discrete points.

[0032] Specifically, it is preferably recommended to first perform voxel grid filtering or statistical outlier removal to reduce the data volume and initially remove noise, and then combine with a denoising algorithm based on a geometric model for further refined processing.

[0033] Step 1.3, Initially screen the images based on the quality of the two-dimensional images. This step can filter out low-quality two-dimensional images to ensure texture quality.

[0034] The embodiment further proposes to initially screen the collected two-dimensional RGB images based on indicators such as sharpness, contrast, illumination uniformity, and noise level.

[0035] Specifically, it is preferably recommended to convert the collected two-dimensional RGB images into grayscale images, measure sharpness through the Laplacian operator, measure contrast through the standard deviation, measure illumination uniformity through the ratio of the standard deviation to the mean, and measure the noise level through the local variance method. According to the influence degrees of the four indicators on the image quality level, set the weights of the four indicators, calculate the image quality level. Then determine the threshold, and initially screen out the images within the threshold range as high-quality images.

[0036] It is preferably recommended to set the weights of the four indicators to 0.4, 0.3, 0.2, and 0.1 in sequence. It is preferably recommended to set the threshold to 0.75. If the calculated is higher than the threshold, then the image is considered a high-quality image.

[0037]

[0038] Among them, is sharpness, is the contrast ratio, is the illumination uniformity, is the noise level, , , , are the corresponding weights.

[0039] Step 1.4: Based on the overlap rate between images, redundant images are removed. This step can reduce data redundancy and improve the computational efficiency of subsequent steps. The overlap rate between adjacent frames can be calculated based on image feature points. If the overlap rate is higher than the threshold, the redundant frame is removed, and finally, the key frames of the images are obtained.

[0040] Specifically, traverse the high-quality images after preliminary screening frame by frame from beginning to end, extract the feature points of the current key frame and the next frame (adjacent frame), and calculate the feature point matching pairs between the two frames. According to the number of matching feature points, calculate the overlap rate between the two frames. If the overlap rate exceeds the threshold, it is considered that the similarity between the two frames is too high, and this adjacent frame is removed, and continue to calculate the overlap rate between the current key frame and the new adjacent frame; if the overlap rate is less than or equal to the threshold, the current key frame is retained, the current adjacent frame is set as the new key frame, and continue to traverse. After the traversal is completed, all the obtained images are the key frames of the two-dimensional RGB images. The key frames can reduce redundant data while ensuring full coverage and improve the computational efficiency of subsequent steps.

[0041] The overlap rate calculation formula proposed by the present invention is as follows:

[0042] Step 2), feature extraction, which is divided into two-dimensional image feature extraction and three-dimensional image feature extraction.

[0043] The preferred implementation method in the embodiment is: Step 2.1, extract two-dimensional image features, which is divided into 3 steps: Step 2.1.1, perform illumination correction and white balance processing on the two-dimensional images screened in Step 1.4.

[0044] Step 2.1.2, align multiple images and splice them into a large-sized panoramic image, and segment teeth (hard tissues) and soft tissues on the panoramic image.

[0045] Step 2.1.3, adopt different algorithms for the tooth (hard tissue) part and the soft tissue part, and extract feature points respectively.

[0046] Preferably, in the embodiment, the U-Net model is selected for improvement, and the labeled two-dimensional images (the tissue regions such as teeth and gums have been manually labeled) are used to train the U-Net model. The U-Net model extracts features from the input image through the encoder, and the decoder of the U-Net model uses these features for pixel-by-pixel classification, outputting the category to which each pixel belongs (such as soft tissue and hard tissue). The cross-entropy loss function is used to optimize the model to minimize the difference between the prediction and the true label. The advantage of the U-Net model is that, with end-to-end training, it can directly learn features from the original image and output high-quality segmentation results; through the skip connection method, it can better maintain the spatial resolution of the image, thus providing a more accurate boundary segmentation, which is suitable for the fine segmentation of hard and soft tissues.

[0047] See Figure 2 , further, the present invention proposes to add an attention mechanism Attention to the U-Net model, and optimize the U-Net architecture through the attention gate, so as to improve the segmentation accuracy. The attention gate has two inputs, one is the feature map from the encoder , and the other is the feature map from the decoder . After convolution and the Sigmoid activation function, an attention map is generated. The generated attention map is multiplied element by element with , and different weights are assigned to each pixel of the decoder feature map, and the regions with lower weights are suppressed. Through the attention gate, the model can pay more attention to the key regions at the junction of hard and soft tissues and reduce the influence of irrelevant regions.

[0048] See Figure 3 , specifically in implementation, the U-Net model part can adopt the existing technology, and in this embodiment, a 4-layer encoder structure is preferably adopted, and a 4-layer decoder structure is correspondingly set.

[0049] Preferably, the classical DPM model (deformable part model) for object detection and shape analysis is selected, soft tissue images are collected, and the positions and quantities of the geometric feature points of the soft tissue are defined. Using these labeled data, the DPM model is trained to learn the local deformation characteristics of the soft tissue. The advantage of the DPM model is that through the feature description of independent components and the deformation model, it can adapt to the morphological changes of soft tissues in different individuals and different situations, and effectively extract the local features of soft tissues.

[0050] Specifically, for the key frames of the screened images, illumination correction is performed through adaptive histogram equalization, and white balance processing is performed through the gray world assumption to further ensure the image quality.

[0051] Specifically, based on the feature points and feature point matching in step 1.4, the first frame of the image is regarded as the initial panoramic image. Subsequently, the subsequent images and the current panoramic image are aligned and stitched using the RANSAC image registration algorithm, and the Laplacian pyramid fusion is used to eliminate the seams. By incrementally fusing and stitching large-scale panoramic images using the Laplacian pyramid, the computational load can be reduced and the stitching efficiency can be improved.

[0052] Specifically, the U-Net semantic segmentation model is used to segment the panoramic dental images to obtain the segmentation masks of teeth (hard tissues) and soft tissues, which facilitates subsequent differential processing of hard and soft tissues.

[0053] Specifically, based on the segmentation mask of teeth (hard tissues), the teeth (hard tissues) part is extracted. After grayscale processing, parameters such as the number of feature points are set, and the ORB feature extraction algorithm is used to extract feature points from the teeth (hard tissues) part.

[0054] Specifically, based on the segmentation mask of soft tissues, the soft tissues part is extracted. The trained DPM model is used to extract feature points from the soft tissues part.

[0055] Step 2.2, extracting 3D point cloud features. Specifically, it is preferably recommended to extract feature points from the point cloud based on PointNet++.

[0056] Step 3, performing offline parametric 3D reconstruction on the soft tissue part.

[0057] Step 3.1, based on the deep learning method of multi-modal data fusion, the feature points of the teeth (hard tissues) and the soft tissue part are respectively matched with the point cloud feature points, and the point cloud is divided into the teeth (hard tissues) part and the soft tissue part.

[0058] Preferably, a Transformer model for cross-modal matching is trained to ensure the accuracy of feature point matching.

[0059] The present invention further proposes that by using the Transformer model, the two-dimensional image features and the three-dimensional point cloud features are fused into a unified feature space. The preferred implementation of feature point matching is as follows: First, the extracted two-dimensional image features and three-dimensional point cloud features are mapped to the same dimension through a linear transformation .

[0060]

[0061]

[0062] Among them, is the two-dimensional image embedding feature vector, is a three-dimensional point cloud embedding feature vector, is a two-dimensional image linear transformation weight matrix, is a three-dimensional point cloud linear transformation weight matrix, is a two-dimensional image feature vector, is a three-dimensional point cloud feature vector.

[0063] Secondly, the features of the two-dimensional image and the three-dimensional point cloud with the same dimension are concatenated to obtain a joint feature vector containing the features of the two modalities .

[0064]

[0065] Use linear transformation to map the joint features to queries (Q), keys (K), and values (V). Use the self-attention mechanism of the Transformer model to calculate the weight matrix, which represents the similarity between different features.

[0066]

[0067] Among them, represents the weight matrix, () represents the normalization function.

[0068] Finally, use cosine similarity to measure the similarity between the two-dimensional image feature points and the three-dimensional point cloud feature points, and find the corresponding feature points.

[0069] By matching the two-dimensional image feature points of the teeth (hard tissue) and soft tissue with the point cloud respectively, the point cloud is divided into two parts of hard and soft tissues, so as to realize the special processing of the soft tissue part in the subsequent steps.

[0070] Step 3.2, convert the point cloud into a voxel grid, and introduce a parameterized distorted voxel grid method for the soft tissue part of the point cloud. Introduce deformation parameters for each voxel in the soft tissue part, and use the translation, rotation, and scaling of each voxel to describe the small deformation of the soft tissue part. Use an optimization algorithm to solve the deformation parameters.

[0071] Preferably, set a reasonable voxel grid size to balance the speed of the optimization process and the accuracy of the optimization result. It is preferably recommended to set the voxel grid size to 0.5 - 1 mm. During specific implementation, it can be flexibly adjusted according to the point cloud density and the required accuracy.

[0072] Specifically, use the VoxelGrid class in the PCL library to voxelize the point cloud, and divide the point cloud into voxel grids of a specified size. Introduce a parameterized distorted voxel grid method. Introduce deformation parameters for each voxel in the soft tissue part to adapt to the deformation of the non-rigid region. The deformation parameters include a translation vector, a rotation matrix, and a scaling factor. Define the position of the deformed voxel as:

[0073] Among them, is the position of the deformed voxel, is the position of the initial voxel, is the translation vector, is a 3×3 rotation matrix, is the scaling factor.

[0074] Define the distance between the voxel position and the corresponding point cloud as the distance error objective function , and by adjusting the deformation of the voxel grid, make each voxel as close as possible to the actual position of the point cloud:

[0075] Among them, the position of a certain voxel in the voxel grid is , and the point in the corresponding point cloud is .

[0076] Define the smoothness of the deformation parameters between adjacent voxels as the smoothness constraint objective function , and by minimizing the difference in deformation parameters between adjacent voxels, prevent local over-twisting or discontinuities of the grid, and ensure the stability and natural deformation of the grid.

[0077]

[0078] Among them, and are the deformation parameters of adjacent voxels i and j in the voxel grid.

[0079] Use the non-linear least squares method to preliminarily solve the deformation parameters.

[0080] Step 3.3, use the clustering algorithm to fuse the soft tissue part into several small regions. Regard each small region as an approximate rigid body and optimize the deformation parameters based on the objective function.

[0081] Specifically, based on the K-Means algorithm, uniformly select the initial cluster centers, and according to the distance between voxels and the deformation parameters of voxels, fuse adjacent voxels of the soft tissue part into several small regions. The fusion process makes the variance of the deformation parameters of voxels in each small region as small as possible, which can further ensure the continuity of the voxel grid deformation and better simulate the natural deformation of soft tissues. Regard each small region as an approximate rigid body and re-initialize the deformation parameters. For each small region, use the non-linear least squares method to optimize the deformation parameters based on the distance error and smoothness constraint objective functions.

[0082] Step 3.4, repeat Step 3.3 for the iterative optimization process. Adopt a segmented fusion method, that is, in each iteration, based on the above principle, locally fuse small regions into several large regions until all the voxel grids of the soft tissue part are fused into one large region. In each iteration, each voxel grid retains translation, rotation, and scaling deformations as the initial input for the next iteration.

[0083] Preferably, set a reasonable segmented fusion step size to balance the speed of the optimization process and the accuracy of the optimization result. It is preferably recommended to set the segmented fusion step size to 2 mm, that is, the side length of the fused large region is about 2 mm longer than that of the small region before fusion. In specific implementation, it can be flexibly adjusted according to the point cloud density and required accuracy.

[0084] Step 4), based on the two-dimensional panoramic image and the triangular mesh model, map the texture pattern onto the three-dimensional oral model.

[0085] Step 4.1, convert the voxel grid model into a triangular mesh model. Map the texture data onto the triangular mesh model.

[0086] Specifically, use the Open3D library to convert the voxel grid model into a triangular mesh model. Based on the feature point matching results in Step 3.1, for each triangular patch, calculate the texture coordinates according to the feature point matching results. Use UV mapping to map the pixel coordinates of the texture image onto the vertices of the triangular mesh.

[0087] Step 4.2, evaluate the texture details of the model based on the color variance, normalize the color variance into a probability value, and generate a probability map. Based on the probability map, set a threshold to retain high-quality texture information.

[0088] Preferably, set a reasonable probability threshold to balance the high quality and continuity of the texture quality. It is preferably recommended that the probability threshold be 0.6 - 0.85. In specific implementation, it can be fine-tuned according to the texture quality requirements. Specifically, for each triangular patch, calculate the color variance of its corresponding texture region , the larger the variance, the richer the texture details.

[0089]

[0090] where is the average color value of the j-th triangular patch, is the color value of the k-th sampling point in the j-th triangular patch, is the number of sampling points in the texture region corresponding to the j-th triangular patch, k is the index of the sampling point in the texture region of the j-th triangular patch, and j is the index of the triangular patch. Normalize the color variance to generate a probability map , represent the degree of detail richness of each texture area using a probability map. Specifically, the following formula can be used for normalization

[0091] where is the maximum value of color variance, is the minimum value of color variance, is the probability value of the j-th triangular patch.

[0092] In specific implementation, a threshold can be set according to the probability map to retain the texture areas with probability values higher than the threshold and remove the texture areas with probability values lower than the threshold, so as to quickly obtain a high-quality and detail-rich texture mapping result.

[0093] Step 4.3, perform local color adjustment and global color adjustment on the filtered texture information to obtain a globally uniform model texture mapping result.

[0094] Preferably, select a suitable color adjustment method according to the lighting conditions and tissue characteristics of the oral cavity environment, and flexibly adjust the weights of local and global color adjustments to finally achieve the best visual effect. For example, for the lighting conditions of local light sources in the oral cavity, use adaptive histogram equalization to enhance the brightness and contrast of local areas while avoiding over-enhancement globally; for the highlight areas of teeth, use brightness stretching to restore details; for soft tissues such as gums, use adaptive contrast enhancement to make the details of soft tissues more obvious while avoiding overexposure. It is preferably recommended that when the global lighting is uneven, appropriately increase the weight of global color adjustment to ensure the overall brightness and contrast of the image; in areas with rich details, appropriately increase the weight of local color adjustment to enhance the detail performance of these areas.

[0095] In specific implementation, those skilled in the art can use software technology to realize the automatic operation of the above process. Correspondingly, if an oral three-dimensional model texture mapping scheme for soft tissues is provided, including a computer or a server, and the above process is executed on the computer or the server for oral three-dimensional model texture mapping of soft tissues, it should also be within the protection scope of the present invention.

[0096] In another embodiment, an electronic device is also involved, including at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the above-mentioned oral three-dimensional model texture mapping method for soft tissues.

[0097] In another embodiment, there is also provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the above-described method for texture mapping of a three-dimensional oral cavity model of soft tissue.

[0098] In another embodiment, there is also provided a computer program product including a computer program, characterized in that when the computer program is executed by a processor, it implements the above-described method for texture mapping of a three-dimensional oral cavity model of soft tissue.

[0099] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without creative effort. Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course also by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disc, etc., including several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments. The above is only the specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, and all should be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. An oral three-dimensional model texture mapping method for soft tissues, characterized in that, It includes the following processes: Scanning and collecting two-dimensional images and three-dimensional point cloud data of the oral cavity; Performing panoramic fusion and hard and soft tissue segmentation processing on the two-dimensional images; For the separated soft tissue part, an offline parameterization method is adopted, and the three-dimensional model of the soft tissue is reconstructed by setting deformation parameters and based on iterative optimization of segmented fusion; Mapping the texture information of the two-dimensional image to the three-dimensional model, and optimizing and screening the texture information through a probability map to complete the texture mapping of the three-dimensional model of the oral soft tissue.

2. The method according to claim 1, characterized in that: When performing data collection, noise filtering and redundant frame removal are performed on the three-dimensional point cloud data, and the redundant frame removal is based on the overlap rate between adjacent frames.

3. The method according to claim 1, wherein: The hard and soft tissue segmentation adopts an improved U-Net model. An attention mechanism is added between the encoder and the decoder of the U-Net model. The boundary region between the hard and soft tissues is enhanced through an attention gate, and a segmentation mask is output.

4. The method according to claim 3, wherein: The feature point extraction of the soft tissue adopts a deformable part model, the feature point extraction of the hard tissue adopts the ORB algorithm, and the two-dimensional image features and the three-dimensional point cloud features are matched through a cross-modal Transformer model.

5. The method according to claim 1, wherein: When performing offline parameterized three-dimensional reconstruction, the soft tissue point cloud is converted into a voxel grid, translation, rotation, and scaling deformation parameters are defined for each voxel, and the deformation parameters are solved by optimizing the objective function.

6. The method according to claim 5, wherein: Optimizing the voxel grid includes dividing the soft tissue into several small regions based on the K-Means algorithm, treating each region as an approximately rigid body for segmented fusion iterative optimization until all voxels are fused into a whole.

7. The method according to claim 1, characterized in that: The texture mapping includes calculating the color variance of the texture region corresponding to the triangular patch, generating a probability map and retaining the texture region with a probability value higher than the corresponding threshold, and performing local and global color adjustment processing.

8. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that: When the processor executes the program, it implements the method for texture mapping of the three-dimensional model of the oral cavity for soft tissue as described in any one of claims 1 to 7.

9. A non-transitory computer-readable storage medium storing a computer program thereon, characterized in that: When the computer program is executed by the processor, it implements the method for texture mapping of the three-dimensional model of the oral cavity for soft tissue as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that: When the computer program is executed by the processor, it implements the method for texture mapping of the three-dimensional model of the oral cavity for soft tissue as described in any one of claims 1 to 7.

Images