Tooth colorimetric intelligent detection method based on shade guide twinborn deep learning

Abstract

The invention relates to a tooth colorimetric intelligent detection method based on shade guide twinborn deep learning, and belongs to the field of tooth colorimetric intelligent detection. The method comprises the following steps: acquiring an image of a standard color tooth and a natural tooth image in a shade guide, classifying according to the category of the tooth color, and constructing a sample data set; meanwhile, the tooth region segmentation image is marked, and a tooth segmentation data set is constructed; based on the sample data set, constructing and training a Siamese-MaxViT twin network model to obtain a tooth color difference model; based on the tooth segmentation data set, constructing and training a YOLOv11 segmentation model to obtain a tooth region segmentation model; and based on the tooth region segmentation model and the tooth color difference model, performing tooth segmentation and colorimetric detection on the to-be-detected tooth image. The method can be widely applied to medical scenes such as dental image segmentation preprocessing and intelligent tooth colorimetry, and has the advantages of high detection precision, high adaptability, high visualization degree and the like.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of intelligent detection of tooth color comparison, and particularly relates to a tooth color comparison intelligent detection method based on color comparison plate twin deep learning. BACKGROUND

[0002] In recent years, with the continuous improvement of technological level, digital image processing and machine vision technology gradually penetrates into the medical field and shows important application value in oral repair. Dental color comparison technology, as a crucial link in the process of oral repair, its accuracy directly affects the naturalness and aesthetics of the restoration. At the same time, with the improvement of people's living standards and the enhancement of aesthetic consciousness, the public's attention to oral health and tooth appearance is growing. The accuracy of color comparison not only relates to the visual coordination of the restoration effect, but also reflects the fine, standardized and professional level of modern oral medical service. Therefore, carrying out high-precision dental color comparison research has important practical significance and development prospects for promoting the continuous progress of oral repair technology, improving the quality of medical services and meeting the individualized aesthetic needs.

[0003] Currently, dental shade matching methods mainly include traditional visual shade matching and digital shade matching. Traditional visual shade matching is the most common method, which mainly relies on the experience of doctors to compare the color of the patient's teeth with the shade guide (such as VITA Classical, VITA 3D-Master, etc.). However, this method has a large subjective error, and is easily affected by light source environment, observation angle and eye fatigue. Therefore, traditional visual shade matching has a large subjective error, and currently relies on experienced doctors, but even so, there may be large differences in shade matching results between different doctors. Therefore, traditional visual shade matching is difficult to standardize and can only be used as a basic method for clinical shade matching. Digital shade matching instruments are gradually applied in oral clinics, including spectrophotometers, colorimeters, spectrometers and digital cameras. However, digital shade matching devices can improve the accuracy of shade matching, but still have certain limitations. For example, there are differences in measurement results between different devices, some devices require strict light control, and the price is high, limiting their application in primary medical institutions. In addition, these devices are mainly used to measure the color parameters of teeth, and cannot intelligently analyze complex tooth color distribution patterns, so doctors still need to manually compare and make decisions, which is difficult to completely replace visual shade matching. In recent years, the rapid development of machine vision technology has brought revolutionary breakthroughs to dental shade matching. The shade matching technology combines deep learning and image processing algorithms to analyze a large amount of tooth color data and establish a standardized shade matching model. Studies have shown that deep learning-based shade matching systems are superior to traditional methods in color recognition and color matching accuracy, which can reduce human error and improve the consistency of shade matching. However, there are no widely used deep learning shade matching products on the market at present, and most researches are still in the laboratory stage, lacking optimization for actual clinical needs. In particular, in terms of combination with international standards (such as VITA shade guide), there are still many technical challenges. SUMMARY

[0004] The tooth color intelligent detection method based on the color chart Siamese deep learning aims to provide a tooth color intelligent detection method based on a color chart Siamese deep learning.

[0005] To achieve the above purpose, the technical scheme of the present application is: a tooth color intelligent detection method based on a color chart Siamese deep learning, comprising:

[0006] Collecting images of standard color teeth in a color chart and natural tooth images, and classifying them according to the categories of tooth colors to construct a sample data set; at the same time, labeling tooth region segmentation images to construct a tooth segmentation data set;

[0007] Based on the sample data set, a Siamese-MaxViT twin network model is constructed and trained to obtain a tooth color difference model; based on the tooth segmentation data set, a YOLOv11 segmentation model is constructed and trained to obtain a tooth region segmentation model;

[0008] Based on the tooth region segmentation model and the tooth color difference model, the tooth segmentation and color detection of the tooth image to be inspected are performed.

[0009] Further, the specific steps of constructing the sample data set and the tooth segmentation data set are as follows:

[0010] S11, in a dark room, using a D65 light source, placing a VITA color chart on a gray background plate, using a macro focus lens digital SLR camera to shoot high-definition images of each color number of the VITA color chart, ensuring consistent illumination and angle; using an Easyshade colorimeter to read the Lab values of each color number, measuring each region of 1 / 3 of the neck, 1 / 3 of the middle and 1 / 3 of the cut three times, taking the average value and recording;

[0011] S12, use the oral half hook to pull the upper lip of the patient, place the gray background plate behind the maxillary anterior teeth, and shoot under the same conditions as S11; collect natural tooth images of patients of different ages, genders, and races, use the VITA color chart to color, and record the color number; use the Easyshade colorimeter to measure the Lab value of the natural tooth, measure three times in the neck 1 / 3, middle 1 / 3, and incisal 1 / 3 regions, take the average value and record;

[0012] S13, cut the high-definition images of each color number of the VITA color chart and the natural tooth images, label the VITA color number, Lab value and tooth region segmentation information, construct a sample data set of 1600, divide the training set, validation set and test set in the ratio of 8:1:1, and construct a tooth segmentation data set of 1500, divide the training set, validation set and test set in the ratio of 8:1:1.

[0013] Further, the Siamese-MaxViT twin network model adopts the MaxViT backbone network based on the Transformer architecture as the feature extraction module of the twin neural network, and also introduces an adaptive dynamic selection loss function ADL.

[0014] Further, the Siamese-MaxViT twin network model is composed of two sub-networks sharing weights.

[0015] Further, the MaxViT backbone network includes a 3x3 convolution layer, a plurality of MaxViT-Blocks of different levels connected in turn from input to output; the MaxViT-Block is composed of an MBConv module, a BlockAttention self-attention module and a GridAttention self-attention module.

[0016] Further, the MBConv module includes a depthwise separable convolution, a pointwise convolution and an SE module, in addition, the MBConv module also introduces a residual structure, the BlockAttention self-attention module and the GridAttention self-attention module, which respectively perform self-attention mechanism on two different scales of blocks and grids, realizing multi-angle processing of local and global information of the image; the Block Attention self-attention module divides the image into a plurality of regional blocks, the features between these blocks are calculated through the self-attention mechanism to obtain the attention degree of each block to other blocks and obtain the corresponding weight, learning the dependency relationship between each block in the image; the GridAttention self-attention module captures the global context information in the image by performing self-attention calculation in the entire grid region.

[0017] Further, the adaptive dynamic selection loss function ADL is an adaptive weight loss function combining contrastive loss and center loss.

[0018] Furthermore, the definition of the contrast loss function is as follows:

[0019]

[0020] Where D represents the Euclidean distance between samples x1 and x2 in the feature space, m is a marginal constant used to control the minimum allowable distance between dissimilar samples, and y is the relationship label between the two samples. If the two samples belong to the same category, their relationship label is 1, and the contrast loss value becomes D 2 If two samples belong to different categories, their relationship label is 0, and the contrast loss value at this time becomes 0.5×max(0,mD) 2 ; Therefore, for similar sample pairs, the goal of the contrast loss function is to minimize the distance between their feature vectors. The smaller the distance, the smaller the loss value. For sample pairs of different categories, the goal of the contrast loss function is to ensure that the distance between their feature vectors is greater than the boundary value m. If the distance between them is less than the boundary value, the loss function value will increase, forcing the model to separate them.

[0021] Furthermore, the central loss function is defined as follows:

[0022]

[0023] Among them, f(x i ) is the sample x i The feature representation of is the sample feature mean for each category, and y is the i Sample x i , where the center loss function is defined as half of the square of the Euclidean distance between the sample feature vector and its category center. In each training step, the category center is continuously updated, and the distance between each sample in the same category and its category center is continuously reduced, thereby strengthening the features of all similar samples to gather towards the category center; therefore, samples of different categories can not only be correctly classified in the feature space, but also features within the same category are more compactly distributed in space, which helps the model learn more discriminative vector features;

[0024] The adaptive weight loss function is defined as follows:

[0025] L adapt =L contrastive +λL center

[0026] λ is a learnable hyperparameter used to dynamically adjust the weights of contrast loss and center loss to ensure that they can jointly optimize learning and prevent the model from overfitting or underfitting during training.

[0027] The application further provides a computer readable storage medium, which stores computer program instructions capable of being run by a processor, and when the processor runs the computer program instructions, steps of any of the above methods can be implemented.

[0028] Compared with the prior art, the application has the following beneficial effects: the application adopts YOLOv11 to achieve higher segmentation accuracy under a smaller model, meeting the segmentation task of tooth images; secondly, the classification adopts a MaxViT backbone network based on a Transformer architecture as a feature extraction module of a twin neural network, so as to fully combine the local feature extraction capability of a CNN and the global information modeling capability of a Transformer structure, thereby improving the expression capability and discrimination capability of the model in tooth color difference detection. The inputs of the subnetworks of the twin neural network are respectively a standard color card and a color image segmented from a tooth to be inspected, after the images are input into the model, the MaxViT feature extraction backbone is used to capture high-level features of tooth colors, and then more discriminative feature representations are extracted. The MaxViT uses a multi-axis window attention mechanism to extract local low-level information under the premise of effectively reducing the computational complexity, and combines grid attention to model global features of the entire image, so that the model can accurately capture the slight differences of tooth colors in a multi-scale space, and effectively improve the detection accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 VITA color chart category.

[0030] Figure 2 The tooth intelligent color matching detection model designed by the application.

[0031] Figure 3 MaxViT feature extraction backbone network structure.

[0032] Figure 4 The tooth intelligent color matching detection method flow chart of the application based on color chart twin deep learning.

[0033] Figure 5 PubFig, fingerprintv2, VITA dataset partial image examples.

[0034] Figure 6 Tooth color difference detection model ablation experiment training accuracy graph.

[0035] Figure 7 Different feature extraction Backbone training accuracy and verification Loss curve under the twin network architecture.

[0036] Figure 8 Experiment precision and validation loss curve for PubFig dataset.

[0037] Figure 9 Experiment precision and validation loss curve for fingerprintv2 dataset. DETAILED DESCRIPTION

[0038] The technical solutions of the present application will be described in detail below with reference to the drawings.

[0039] The present application provides a tooth color matching intelligent detection method based on color chart twin deep learning, comprising:

[0040] Collect images of standard color teeth in the color chart and natural tooth images, and classify them according to the categories of tooth color to construct a sample data set; at the same time, label the tooth region segmentation image to construct a tooth segmentation data set;

[0041] Based on the sample data set, a Siamese-MaxViT twin network model is constructed and trained to obtain a tooth color difference model; based on the tooth segmentation data set, a YOLOv11 segmentation model is constructed and trained to obtain a tooth region segmentation model;

[0042] Based on the tooth region segmentation model and the tooth color difference model, tooth segmentation and color matching detection are performed on the to-be-tested tooth image.

[0043] The following is the specific implementation process of the present application.

[0044] The present application provides a tooth color matching intelligent detection method based on color chart twin deep learning, mainly comprising:

[0045] 1. Color chart acquisition and data set construction

[0046] The VITA color chart in the application is from a university-affiliated stomatological hospital, and there are a total of sixteen types of color charts. The image of the tooth color chart is simulated as follows: (1) In a dark room, use D65 light source, place the color chart on the gray background plate, use a macro focus lens digital SLR camera to shoot the high-definition image of each color number of the VITA color chart, and ensure the consistency of illumination and angle. Use Easyshade colorimeter to read the Lab value of each color number, measure three times in the neck 1 / 3, middle 1 / 3 and incisal 1 / 3 regions respectively, take the average value and record; (2) Use the oral half-lap hook to pull the upper lip of the patient, place the gray background plate behind the maxillary anterior teeth, and the shooting conditions are the same as 1. Collect the images of natural teeth of patients of different ages, genders and races, and use the VITA color chart to color by experts with more than 10 years of repair experience, and record the color number. Use Easyshade colorimeter to measure the Lab value of the natural teeth, measure three times in the neck 1 / 3, middle 1 / 3 and incisal 1 / 3 regions respectively, take the average value and record; (3) Cut the image, label the VITA color number, Lab value and tooth region segmentation information. In summary, the VITA color chart dataset has a total of 1600 images, and the training set, validation set and test set are divided according to the ratio of 8:1:1. Therefore, there are 80 images for each color to construct the color chart dataset for training the MaxViT network, and there are 10 images for each color for verification and testing. The dataset for segmenting the tooth region has a total of 1500 images, which are divided into training set, validation set and test set according to the ratio of 8:1:1, that is, 1200:150:150, which is used for segmentation of YOLOv11. The 16 types of VITA color charts are as shown in Figure 1 . .

[0047] 2. Color difference test method and model architecture

[0048] In the intelligent tooth color detection task, the color of the tooth to be tested usually presents a relatively uniform distribution. When there is a color deviation, it often presents a large-scale overall change rather than a local pixel-level anomaly. In addition, different tooth colors may have different reflection effects under different environmental conditions, which may also cause subtle changes in color at different location areas. Therefore, to determine whether the tooth is similar to the standard color card, more attention needs to be paid to the color information in the entire image area, so as to more comprehensively learn the color information and thus improve the robustness and generalization ability. The traditional Siamese neural network usually relies on CNN as a feature extractor to learn the deep features of tooth color. However, CNN uses local convolution operation, and its receptive field is very limited, which is difficult to effectively model the global color distribution and complex texture information of the tooth. At the same time, when facing a small color deviation, CNN cannot capture these key information, and is prone to have insufficient generalization ability. In order to solve the above problems, the application innovatively uses a backbone network MaxViT (Maximum Vision Transformer) based on the Transformer architecture as the feature extraction module of the Siamese neural network, so as to fully combine the local feature extraction ability of CNN and the global information modeling ability of the Transformer structure, thereby improving the expression ability and discrimination ability of the model in tooth color difference detection. MaxViT uses a multi-axis window attention mechanism to effectively reduce the computational complexity, uses window attention to extract local low-level information, and combines grid attention to model the global features of the entire image, so that the model can accurately capture the small differences in tooth color in a multi-scale space, effectively improving the detection accuracy. In addition, in order to further optimize the similarity measurement in the feature space, the application also introduces an adaptive dynamic selection loss function (ADL) based on the Siamese-MaxViT structure, which aims to expand the distance between different class samples, and make the feature distribution of the same class more compact, so as to obtain more stable color features. The overall structure of the model is as shown in Figure 2 The Siamese-MaxViT model is composed of two sub-networks sharing weights, wherein the inputs of the sub-networks are respectively a standard color card and a tooth color image to be tested. After the image is input into the model, the MaxViT feature extraction backbone is used to capture the high-level features of the tooth color, and then a more discriminative feature representation is extracted.

[0049] 3. Feature extraction backbone network

[0050] At present, the deep learning feature extraction backbone is roughly divided into two kinds according to its architecture, namely, based on convolutional neural network and based on Transformer. The former mainly relies on the sliding of the convolution kernel to process local area information, layer by layer extracting features and gradually expanding the receptive field. The neural network of the latter is to convert the input image into a vector through the embedding layer, and then divide it into multiple patches and flatten it as input. The core is to give each element an attention weight through the self-attention mechanism, and the weight information not only depends on itself, but also relates to other feature elements in the sequence. Therefore, the neural network of the Transformer architecture can better capture global information. MaxViT (Multi-Axis Vision Transformer) based on the Transformer architecture not only retains the advantages of the Transformer architecture, but also combines the advantages of the convolutional neural network, fully solving the problems of low computational efficiency and weak local feature extraction capability in image processing. The MaxViT feature extraction backbone network structure is as follows Figure 3The input image is first processed by a 3x3 convolutional layer to extract features and reduce the image size, and then processed by multiple MaxViT-Blocks to extract deeper features. This network architecture, which first extracts local features using a convolutional layer and then captures global information using a Transformer, makes MaxViT efficient and maintains good performance when processing large-scale images. This design is also reflected in the MaxViT-Block, which consists of an MBConv, a Block Attention, and a Grid Attention. The MBConv module combines deep separable convolution, pointwise convolution, and SE (Squeeze-and-Excitation) modules. It inherits the idea of MobileNet, a convolutional neural network, and uses deep separable convolution to reduce the computational complexity of traditional convolution operations. It then combines the features of each channel using pointwise convolution to generate a new feature map. Finally, it uses the SE module to perform global average pooling and feature re-labeling to enhance important channel features and suppress unimportant ones. In addition, the MBConv module also introduces a residual structure to avoid the problem of gradient vanishing during model training. Based on this design, the MBConv module improves feature expression while reducing model computational complexity and optimizing computational efficiency. Another highlight of the MaxViT-Block is its multi-axis attention mechanism, which includes two self-attention modules: BlockAttention and GridAttention. These two modules perform self-attention mechanisms at different scales, allowing the image to process local and global information from multiple angles. BlockAttention divides the image into multiple regional blocks, and the features between these blocks are calculated using a self-attention mechanism to determine the attention degree of each block to other blocks and obtain the corresponding weights, thereby learning the dependency between blocks in the image. GridAttention captures global context information in the image by performing self-attention calculations on the entire grid region. Compared to local blocks, grids focus more on global information in the entire image. Therefore, in the MaxViT-Block, BlockAttention captures the detailed relationships between local regions, while GridAttention extends local information to global information to extract global information. This combination effectively combines information at different scales, enhancing the model's feature expression and generalization capabilities.

[0051] 4. Adaptive dynamic selection of loss function

[0052] In twin neural networks, selecting a suitable loss function is crucial for the optimization of the task. Commonly used loss functions include contrastive loss, cosine similarity loss, triplet loss, center loss, etc. Contrastive loss learns and trains by minimizing the distance between similar samples and maximizing the distance between dissimilar samples; cosine similarity loss optimizes model performance by maximizing or minimizing the cosine similarity between two samples; triplet loss mainly considers the relationship between anchor samples, positive samples and negative samples to learn similarity, and its learning strategy is to make the distance between anchor samples and positive samples smaller and the distance between anchor samples and negative samples larger; center loss learns the center of each class and makes the features of each sample close to the center of the corresponding class, thereby enhancing the aggregation of the same class. However, a single loss function may not meet the needs of actual tasks, so the present application proposes an adaptive weight loss function combining contrastive loss and center loss, aiming to optimize the distance measurement between samples and improve the aggregation ability of samples of the same class.

[0053] The definition of the contrastive loss function is usually as shown in formula (1), wherein D represents the Euclidean distance of samples x1 and x2 in the feature space, m is a marginal constant used to control the minimum allowed distance between dissimilar samples. y is the relationship label between two samples, if two samples belong to the same class, the relationship label is 1, then the contrastive loss value becomes D 2 half, if two samples belong to different classes, the relationship label is 0, at this time the contrastive loss value becomes 0.5*max(0,m-D) 2 . Therefore, when it comes to similar sample pairs, the goal of the contrastive loss function is to minimize the distance between their feature vectors, and the smaller the distance, the smaller the loss value. When it comes to different class sample pairs, the goal of the contrastive loss function is to ensure that the distance between their feature vectors is greater than the boundary value m. If the distance between them is less than the boundary value, the loss function value will increase, thereby forcing the model to separate them.

[0054]

[0055] The definition of the center loss function is usually as shown in formula (2), wherein f(x i ) is the feature representation of sample x i , is the average of the sample features of each class. For each class, the sample x i of y iwhere the center loss function is defined as half of the squared Euclidean distance between the sample feature vector and its class center. In each training step, the class center is constantly updated, and the distance between each sample in the same class and its class center is constantly reduced, thereby strengthening the feature of all samples in the same class to the class center. Therefore, not only can the samples of different classes be correctly classified in the feature space, but also the features in the same class are more compactly distributed in the space, which helps the model to learn more discriminative vector features.

[0056]

[0057] Based on the above analysis, the definition of the adaptive weight loss function proposed in the present application is shown in formula (3).

[0058] L adapt = L contrastive + λL center (3)

[0059] The contrast loss function helps the model to learn effective feature representations that distinguish different classes by minimizing the distance between similar sample pairs and increasing the distance between non-similar sample pairs; the center loss helps the model to focus on learning feature representations within the same class. Through such a combination, the model can further improve the aggregation of samples within the same class while ensuring reasonable inter-sample distance, reduce the overlap between classes, and thus improve the feature learning ability and recognition performance of the model. Moreover, since the scale of the contrast loss and the center loss differs when calculating the distance, the model may dominate the entire optimization process during learning. Therefore, when combining the two loss functions, a learnable hyperparameter λ is introduced to dynamically adjust the weights of the contrast loss and the center loss, ensuring that they can jointly optimize learning and prevent overfitting or underfitting of the model during training, thereby improving the accuracy and stability of the model.

[0060] 5. Specific implementation process and steps

[0061] The process of the tooth intelligent colorimetric detection method based on color chart twin deep learning is shown in Figure 4 Fig. 1. First, the model is trained: input the colorimetric detection dataset into the segmentation model of YOLOv11 for training; input the VITA color chart dataset into the MaxViT classification network for training to obtain the finally trained overall network system. After completing the model training, the test set is input into the twin network to judge the performance of the model. The twin network model outputs the similarity compared with all color charts, and outputs the most similar color to the color chart through the comparison of the similarity.

[0062] 6. Embodiment example

[0063] 6.1 Experimental environment + dataset construction introduction

[0064] The present experiment respectively selects the face recognition public dataset PubFig and the fingerprint matching public dataset fingerprintv2 and the tooth color difference test image dataset VITA constructed by the present application to verify and cross compare the method, wherein the related information of the PubFig dataset, the fingerprintv2 dataset and the Color-check dataset is shown in Table 1, and the corresponding part of the image is shown in the table. Figure 5 In addition, the hardware and software configuration of the experimental environment related in this section is shown in Table 2.

[0065] Table 1 Dataset related information

[0066]

[0067]

[0068] Table 2 Experimental environment setting

[0069]

[0070] The input image size of the method is 512x512, the adaptive gradient optimizer (initial learning rate is set to 1e-4) is used to optimize the model parameters, and the StepLR learning rate decay strategy is also used to decay the learning rate by x0.97 every epoch, the training round epoch is set to 100, and the batchsize size is set to 16. In order to more objectively evaluate the performance of the method, this section selects the test accuracy Precision and AUC value as the color difference test performance index. Among them, the Precision value represents the ratio of the number of correct test samples to the total number of test samples, the higher the Precision value, the more accurate the model test. The AUC value refers to the area under the curve obtained by plotting the false positive rate and the true positive rate of the model, the closer the area is to 1, the stronger the model recognition ability. The corresponding calculation process is shown in formula (4) and formula (5)-(7):

[0071]

[0072] wherein in formula (4), N represents the total number of samples participating in the test, and Tn represents the number of samples correctly testing the color difference relationship. In formula (5), TPR represents the true positive rate, that is, the proportion of samples that are actually positive samples and are identified as positive by the model; FPR represents the false positive rate, that is, the proportion of samples that are actually negative samples and are incorrectly identified as positive by the model. In formulas (6)-(7), TP represents the true positive, that is, the number of positive samples correctly predicted by the model; FP represents the false positive rate, that is, the number of negative samples incorrectly predicted by the model; TN represents the true negative, that is, the number of negative samples correctly predicted by the model; and FN represents the false negative, that is, the number of positive samples incorrectly predicted by the model.

[0073] 6.2 Experimental results and analysis

[0074] Experiment 1: Feature extraction backbone network and adaptive dynamic loss function ablation experiment.

[0075] To verify the effectiveness of the network structure design in the tooth color difference test method based on the Siamese MaxViT network, this section takes Siamese-Res18 as the experimental baseline, replaces the MaxViT feature extraction backbone and the adaptive dynamic loss function ADLoss respectively to perform ablation experiments, and the experimental results are shown in Table 3, and the training process is shown in Figure 6 From the data in Table 3, it can be found that the baseline model performs well in handling the tooth color difference test task, but there is still room for improvement. By replacing the feature extraction backbone from ResNet-18 to MaxViT, the model's Precision and AUC are improved by an average of 2.28% and 1.69%, respectively. At the same time, by replacing the original contrast loss function with the adaptive dynamic selection loss function ADLoss proposed in this application, the model's Precision and AUC are improved by an average of 1.41% and 0.93%, respectively. Finally, experiments are conducted on Siamese-MaxViT and ADLoss. According to the results in the table, the Precision and AUC of the baseline model are improved by an average of 3.74% and 3.68%, respectively. Through the above data analysis, it can be shown that the Siamese-MaxViT feature extraction backbone and the ADLoss loss function proposed in this application are effective in the tooth color difference test task. In addition, from the ablation experiment, it can be found that the test performance of Siamese-MaxViT after replacing the ADLoss loss function is not a simple additive improvement of module stacking, but a mutual stimulation improvement, which further illustrates the effectiveness of the network structure design of the model proposed in this application, and lays a certain foundation for the realization of the tooth color difference test task.

[0076] Table 3 Ablation experiment results of tooth color difference test model

[0077]

[0078] Experiment 2: Comparison experiment of different feature extraction backbones based on Siamese neural network architecture.

[0079] To further verify the advantages of the Siamese-MaxViT model compared with other mainstream feature extraction backbones combined with the Siamese network architecture, this section selects four CNN convolutional neural networks and Transformer feature extraction networks on the Color-check dataset for comparison experiments, and the experimental results are shown in Table 4, and the training iteration process is shown in Figure 7 From Table 4, it can be seen that the feature extraction backbone network based on the Transformer architecture in the Siamese neural network architecture is mostly better than the CNN feature extraction backbone network, which not only shows the effectiveness of the Transformer architecture feature extraction backbone in the tooth color difference verification task, but also shows the advancement of the Siamese-MaxViT feature extraction network architecture proposed in this application.

[0080] Table 4 Comparison experiment of different feature extraction backbones

[0081]

[0082]

[0083] In addition, among the CNN neural network backbones, the ResNeXt model has the best verification performance, with a verification accuracy of 96.97%, while among the Transformer backbones, the MaxViT model performs best, with a verification accuracy of 98.97%, which is 2% and 0.78% higher than the ResNeXt and RMT models, respectively, and its training Train_Loss convergence rounds are lower than those of the two, which shows that the Siamese-MaxViT model can quickly and effectively extract key features from input data for the task, and also further shows the superiority of the Siamese-MaxViT network architecture.

[0084] Experiment 3: Generalization performance experiment of Siamese-MaxViT color difference verification model.

[0085] In order to further evaluate the generalization and robustness of the model proposed in this paper, ResNeXt, ViT, Swin-Transformer, RMT and the model of the present invention with relatively good performance in the comparative experiments are selected in this section, and generalization verification experiments are carried out on the PubFig and fingerprint v2 public datasets respectively. The experimental results are shown in Table 5. The training iteration process is as follows: Figure 8 、 9 As shown in Table 5, Precision-1 and Precision-2 represent the model's verification accuracy on the PubFig and fingerprint v2 datasets, respectively; Loss Convergence Round-1 and Loss Convergence Round-2 represent the model's verification loss on the PubFig and fingerprint v2 datasets, respectively. The experimental results in Table 5 show that the feature extraction backbone, which performs well on the Color-check dataset, also has good generalization performance on both the PubFig and fingerprint v2 datasets. The proposed model achieves recognition accuracies of 89.34% and 88.93% on these two datasets, respectively. Its performance on the PubFig dataset is superior to that of other models, and its performance on the fingerprint v2 dataset is second only to the RMT network. This indicates that although the generalization performance of the proposed model does not completely surpass other models, it is still competitive with other models in multiple datasets. In addition, it can be seen from the training iteration process diagram that the model proposed in the present invention can achieve a higher test accuracy in fewer training rounds, which shows that the model has a strong feature extraction capability and good generalization performance, and can quickly learn effective discriminant features during the training process, thereby improving the overall training efficiency. At the same time, by comparing the convergence curve of the verification loss, it can be seen that the model of the present invention shows a more stable Loss decline trend during the training process, with smaller fluctuations and a more stable overall convergence process, which further illustrates that the model training process is relatively smooth, not prone to falling into local optimality or gradient oscillation, and the model training difficulty is relatively low. While maintaining high accuracy, it also shows better robustness and can adapt to more complex or changing input data, thereby further highlighting the performance advantages of the model proposed in the present invention.

[0086] Table 5 Generalization performance experiment of tooth color difference test model

[0087]

[0088]

[0089] To further verify the effectiveness of the method, the present application selects 100 positive sample pairs and negative sample pairs for experiment, wherein the positive sample pair refers to the standard color card for difference test and the test tooth image belongs to one category without color difference, and the negative sample pair refers to the standard color card for difference test and the test tooth image belongs to a similar category without color difference. Since a large number of color defect samples do not appear in actual life, in this experiment, similar color images are selected as substitutes for color defect samples for experiment, and the experimental results are shown in Table 6. In the table, P is the similarity of the sample pair. From the data in the table, it can be seen that the tooth color difference test method proposed by the present application has a high test accuracy when testing positive sample pairs and negative sample pairs. Among them, the number of positive sample pairs with a similarity P greater than or equal to 0.9 is 97, and the number of negative sample pairs with a similarity less than 0.5 is 55, and the number of negative sample pairs with a similarity greater than 0.5 and less than 0.9 is 41. This shows that the tooth color difference test method based on twin MaxViT proposed by the present application can accurately and effectively test the color difference of teeth.

[0090] Table 6 Tooth color test results

[0091]

[0092] The present application proposes a tooth color difference detection method based on YOLOv11 segmentation and twin Transformer, which can effectively classify the color difference between teeth and standard color cards, and realize quantitative judgment and intuitive presentation of the difference with the standard color card. The method first uses the YOLOv11 network to accurately segment the tooth image, ensuring that the subsequent analysis focuses on the effective area and improving the overall detection efficiency and accuracy; then an improved twin Transformer architecture is introduced, combined with a Siamese-MaxViT feature extraction backbone network, to enhance the sensitivity and discrimination ability of the model to subtle color differences in images. In addition, the present application designs an adaptive dynamic loss function that can adjust the loss weight according to the color difference between samples, improving the model convergence speed and discrimination robustness; and combines a color difference test strategy to convert the detection error into a quantitative indicator, improving the explainability of intelligent tooth color testing. In order to verify the effectiveness and generalization ability of the method, ablation experiments, comparative experiments and generalization performance tests are carried out on the self-built VITA dataset, PubFig face image dataset and fingerprint v2 fingerprint dataset, and the results all show that the method has good precision, robustness and cross-domain adaptability. In summary, the present application can be widely used in dental image segmentation preprocessing, intelligent tooth color matching and other medical scenarios, has the advantages of high detection accuracy, strong adaptability and high visualization, and can effectively reduce the risk of mismatching of the color of the restoration body caused by inaccurate tooth color difference recognition, and improve the accuracy of digital color matching and the quality of oral restoration.

[0093] The application further provides a computer readable storage medium, which stores computer program instructions capable of being run by a processor, and when the processor runs the computer program instructions, steps of any method described above can be implemented.

[0094] The above is the preferred embodiment of the application, and any change made according to the technical solution of the application, as long as the function effect generated does not exceed the range of the technical solution of the application, belongs to the protection scope of the application.

Claims

1. A tooth colorimetric intelligent detection method based on colorimetric twin deep learning, characterized in that: include: Collect images of standard-colored teeth and natural teeth from the color chart, classify them according to tooth color categories, and construct a sample dataset. At the same time, label the tooth region segmentation images and construct a tooth segmentation dataset. Based on the sample dataset, a Siamese-MaxViT twin network model was constructed and trained to obtain a tooth color difference model. Based on the tooth segmentation dataset, a YOLOv11 segmentation model was constructed and trained to obtain a tooth region segmentation model. Based on the tooth region segmentation model and the tooth color difference model, tooth segmentation and colorimetric detection are performed on the tooth image to be inspected.

2. The tooth colorimetric intelligent detection method based on colorimetric twin deep learning according to claim 1 is characterized in that: The specific steps for constructing the sample dataset and tooth segmentation dataset are as follows: S11. In a darkroom, using a D65 light source, place the VITA colorimetric chart on a gray background. Use a digital SLR camera with a macro lens and fixed focus to capture high-definition images of each color number on the VITA colorimetric chart, ensuring consistent lighting and angle. Use an EasyShade colorimeter to read the Lab value of each color number, measuring three times each in the neck 1 / 3, middle 1 / 3, and cut 1 / 3 areas. Take the average value and record it. S12: Use an oral half retractor to pull open the patient's upper lip and place a gray background behind the maxillary anterior teeth. Use the same photography conditions as S11. Collect images of natural teeth from patients of different ages, genders, and ethnicities. Use VITA colorimetric plates to compare colors and record color numbers. Use an Easyshade colorimeter to measure the Lab values ​​of natural teeth. Measure three times each in the cervical 1 / 3, middle 1 / 3, and incisal 1 / 3 regions. Take the average value and record it. S13. Crop the high-definition images of each color number of the VITA shade guide and the natural tooth images, mark the VITA color number, Lab value and tooth area segmentation information, build a sample data set of 1600 images, divide the training set, validation set and test set in a ratio of 8:1:1, build a tooth segmentation data set of 1500 images, divide the training set, validation set and test set in a ratio of 8:1:

1.

3. The tooth colorimetric intelligent detection method based on colorimetric twin deep learning according to claim 1 is characterized in that: The Siamese-MaxViT twin network model uses the Transformer-based backbone network MaxViT as the feature extraction module of the twin neural network, and also introduces the adaptive dynamic selection loss function ADL.

4. The tooth colorimetric intelligent detection method based on colorimetric twin deep learning according to claim 1 or 3, characterized in that: The Siamese-MaxViT twin network model consists of two sub-networks that share weights.

5. The tooth colorimetric intelligent detection method based on colorimetric twin deep learning according to claim 3 is characterized in that: The backbone network MaxViT includes 3×3 convolutional layers connected sequentially from input to output and multiple MaxViT-Blocks at different levels; MaxViT-Block consists of MBConv module, BlockAttention self-attention module, and GridAttention self-attention module.

6. The tooth colorimetric intelligent detection method based on shade plate twin deep learning according to claim 5 is characterized in that: The MBConv module includes depthwise separable convolution, point-by-point convolution, and SE modules. In addition, the MBConv module also introduces a residual structure, BlockAttention self-attention module, and GridAttention self-attention module, which respectively perform self-attention mechanisms at two different scales, block and grid, to achieve multi-angle processing of local and global information of the image; the BlockAttention self-attention module divides the image into multiple regional blocks, and the features between these blocks are calculated through the self-attention mechanism to calculate the degree of attention of each block to other blocks, and obtain the corresponding weights to learn the dependency relationship between each block in the image; the GridAttention self-attention module captures the global context information in the image by performing self-attention calculations on the entire grid area.

7. The tooth colorimetric intelligent detection method based on colorimetric twin deep learning according to claim 3 is characterized in that: The adaptive dynamic selection loss function ADL is an adaptive weight loss function that combines contrast loss and center loss.

8. The tooth colorimetric intelligent detection method based on shade plate twin deep learning according to claim 7 is characterized in that: The definition of the contrast loss function is as follows: Where D represents the Euclidean distance between samples x1 and x2 in the feature space, m is a marginal constant used to control the minimum allowable distance between dissimilar samples, and y is the relationship label between the two samples. If the two samples belong to the same category, their relationship label is 1, and the contrast loss value becomes D 2 If two samples belong to different categories, their relationship label is 0, and the contrast loss value at this time becomes 0.5×max(0,mD) 2 ; Therefore, for similar sample pairs, the goal of the contrast loss function is to minimize the distance between their feature vectors. The smaller the distance, the smaller the loss value. For sample pairs of different categories, the goal of the contrast loss function is to ensure that the distance between their feature vectors is greater than the boundary value m. If the distance between them is less than the boundary value, the loss function value will increase, forcing the model to separate them.

9. The tooth colorimetric intelligent detection method based on shade plate twin deep learning according to claim 8, characterized in that: The definition of the center loss function is as follows: Among them, f(x i ) is the sample x i The feature representation of is the sample feature mean for each category, and y is the i Sample x i , where the center loss function is defined as half of the square of the Euclidean distance between the sample feature vector and its category center. In each training step, the category center is continuously updated, and the distance between each sample in the same category and its category center is continuously reduced, thereby strengthening the features of all similar samples to gather towards the category center; therefore, samples of different categories can not only be correctly classified in the feature space, but also features within the same category are more compactly distributed in space, which helps the model learn more discriminative vector features; The adaptive weight loss function is defined as follows: THE adapt =L contrastive +λL center λ is a learnable hyperparameter used to dynamically adjust the weights of contrast loss and center loss to ensure that they can jointly optimize learning and prevent the model from overfitting or underfitting during training.

10. A computer-readable storage medium storing computer program instructions that can be executed by a processor, wherein when the processor executes the computer program instructions, the steps of the method according to any one of claims 1 to 9 can be implemented.

Images