An oral lesion feature quantification system based on an improved yolov8 model

Abstract

The application discloses an oral cavity lesion feature quantification system based on an improved yolov8 model and relates to the technical field of oral image processing.The application embeds an SAD layer in a Neck module of the yolov8 model, accurately locates and segments a micro tissue area of an oral cavity, and improves recall rate and segmentation precision; constructs an oral cavity composite loss function fused with oral cavity anatomy priori, dynamically adjusts weights, and enhances the feature extraction robustness for different patients, tooth positions and scale areas; adopts bidirectional cross-modal attention guided by a lesion priori, fuses images and medical record texts, suppresses background interference, and solves the problem of multiple source information fragmentation; and outputs lesion area, gray mean and other objective continuous values through a quantification model, provides a quantification reference for doctors, and realizes end-to-end automatic feature extraction and quantification.

Description

Technical Field

[0001] This invention relates to the field of oral image processing technology, and in particular to an oral lesion feature quantification system based on an improved YOLOv8 model. Background Technology

[0002] Current oral imaging analysis practices primarily rely on clinicians' visual observation and single-modality imaging data, such as intraoral X-rays or CBCT scans. This traditional approach has the following drawbacks: low analysis efficiency, requiring clinicians to spend a significant amount of time on manual interpretation, thus prolonging patient waiting times; especially for small tissue areas such as early demineralization zones, micro-plaque attachment areas, or occult periapical low-density areas, which are difficult to accurately identify with the naked eye; due to a shortage of professional dentists in primary healthcare institutions, the quality of analysis depends on the doctor's experience level, leading to inconsistent interpretation standards; and when conducting large-scale oral health screenings, the approach faces practical challenges such as cumbersome procedures, limited human resources, and excessively high costs.

[0003] At the technical level, existing target detection models based on the YOLO algorithm can achieve basic tooth localization in oral images. However, when integrating multi-source information such as patients' structured medical records, clinical symptom descriptions, and anatomical features, they are prone to one-sided information utilization. Furthermore, general-purpose multimodal AI models lack specific optimization for the oral healthcare field, and their understanding of the morphological features of small tissue regions in images (such as the edge morphology, grayscale distribution, and texture complexity of low-density areas) is insufficient, making it difficult to accurately extract and quantify these features. In addition, existing methods have limited ability to suppress interference from complex backgrounds such as alveolar bone and gingiva in oral images, resulting in low recall and feature extraction accuracy for small tissue regions, failing to meet the efficiency and accuracy requirements of clinical auxiliary interpretation. Summary of the Invention

[0004] The purpose of this invention is to provide an oral lesion feature quantification system based on an improved YOLOv8 model, in order to improve the technical problems of low recall rate in the detection of early micro-tissue areas (such as early demineralization areas, dental plaque attachment areas, and periapical low-density areas), large background interference from alveolar bone and soft tissues, and difficulty in real-time fusion and processing of multi-source information such as images and medical records, which lead to insufficient efficiency and accuracy in the extraction and quantification of oral lesion imaging features.

[0005] To achieve the above-mentioned objectives, the present invention provides the following technical solution:

[0006] A system for quantifying oral lesion features based on an improved YOLOv8 model, comprising:

[0007] The data acquisition and processing module is used to acquire multimodal oral data and perform preprocessing to generate standard oral images, structured text data, and initial features of patient background text.

[0008] The model building module is used to build a basic YOLOv8 model based on the YOLOv8 model and the SAD attention mechanism.

[0009] The model training module is used to train the basic YOLOv8 model, calculate the tooth crowding penalty term enhancement loss, hard sample focusing enhancement loss and edge enhancement loss, and generate an improved YOLOv8 model.

[0010] The image detection and segmentation module is used to detect and segment standard oral images using an improved YOLOv8 model, generating detection and segmentation information for each candidate region. The detection and segmentation information includes bounding boxes, tooth position numbers, confidence scores, pixel-level segmentation masks, and depth visual features.

[0011] The cross-modal fusion module is used to dynamically fuse structured text data, initial features of patient background text, and detection and segmentation information of each candidate region through a bidirectional cross-attention mechanism to generate multimodal oral lesion image features of each candidate region.

[0012] The quantization module processes the multimodal oral lesion image features of each candidate region through a quantization model to generate corresponding oral lesion image feature parameters. The oral lesion image feature parameters include lesion area, mean gray level, standard deviation of gray level, texture contrast, and edge gradient amplitude.

[0013] This invention improves recall and segmentation accuracy by embedding SAD attention into the Neck module of the YOLOv8 model, thereby accurately locating and segmenting small oral tissue regions. It constructs an oral composite loss function incorporating prior knowledge of oral anatomy, dynamically adjusting weights to enhance the robustness of feature extraction for different patients, tooth positions, and scale regions. Furthermore, it employs bidirectional cross-modal attention guided by lesion priors, fusing images and medical record text to suppress background interference and address the problem of fragmented multi-source information. Finally, it provides doctors with quantitative references by quantifying the model's output of objective, continuous values ​​such as lesion area and mean grayscale, achieving end-to-end automated feature extraction and quantification. Attached Figure Description

[0014] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a system structure diagram in an embodiment of the present invention;

[0016] Figure 2 This is a structural diagram of the basic YOLOv8 model in an embodiment of the present invention;

[0017] Figure 3 This is a detailed structural diagram of the basic YOLOv8 model in an embodiment of the present invention;

[0018] Figure 4 This is a structural diagram of the SAD-1 layer in an embodiment of the present invention;

[0019] Figure 5 This is a training curve diagram of the improved YOLOv8 model in an embodiment of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0021] Please see Figure 1 This embodiment provides a system for quantifying oral lesion features based on an improved YOLOv8 model, comprising:

[0022] The data acquisition and processing module is used to acquire multimodal oral data and perform preprocessing to generate standard oral images, structured text data, and initial features of patient background text.

[0023] The model building module is used to build a basic YOLOv8 model based on the YOLOv8 model and the SAD (ShuffleAttention-Dental) attention mechanism.

[0024] The model training module is used to train the basic YOLOv8 model, calculate the tooth crowding penalty term enhancement loss, hard sample focusing enhancement loss and edge enhancement loss, and generate an improved YOLOv8 model.

[0025] The image detection and segmentation module is used to detect and segment standard oral images using an improved YOLOv8 model, generating detection and segmentation information for each candidate region. The detection and segmentation information includes bounding boxes, tooth position numbers, confidence scores, pixel-level segmentation masks, and depth visual features. Each candidate region corresponds to a bounding box and a segmentation mask.

[0026] The cross-modal fusion module is used to dynamically fuse structured text data, initial features of patient background text, and detection and segmentation information of each candidate region through a bidirectional cross-attention mechanism to generate multimodal oral lesion image features of each candidate region.

[0027] The quantization module processes the multimodal oral lesion image features of each candidate region using a quantization model to generate corresponding oral lesion image feature parameters. These parameters include lesion area, mean gray level, standard deviation of gray level, texture contrast, and edge gradient amplitude.

[0028] The data acquisition and processing module includes the following processes:

[0029] S1-1. Collect multimodal oral data; the multimodal oral data includes image data and text data; the image data includes intraoral endoscopic photographs, periapical radiographs, panoramic radiographs, and CBCT images; the text data includes structured oral medical records, symptom descriptions, and medical history information.

[0030] S1-2. Denoise, normalize, standardize grayscale, and enhance contrast of the image data to generate image processing data. By preprocessing the image data, the resolution and image channels are unified, and the data differences caused by different devices and shooting conditions are eliminated to obtain a standard image that meets the input requirements of the subsequent model.

[0031] S1-3. Perform semantic extraction, entity normalization, and redundancy removal on the text data to generate structured text data and initial features of patient background text.

[0032] Specifically, entity extraction (extracting tooth position numbers, symptom keywords, etc.) and semantic encoding are performed on structured oral medical records and symptom descriptions to map the corresponding text data into a semantic vector sequence, resulting in structured text data. Simultaneously, the patient's medical history information (basic information and past medical history) is extracted separately and converted into fixed-dimensional initial features of the patient's background text through embedding layers or discrete value mapping. In this embodiment, semantic encoding uses a pre-trained language model in the medical field, such as the BioBERT model.

[0033] In achieving precise localization and segmentation of oral lesions, low recall and insufficient segmentation accuracy due to background interference exist when dealing with small lesions such as dental caries, plaque, and early periapical periodontitis. To address this, this embodiment, based on the standard YOLOv8 architecture, inserts a ShuffleAttention-Dental (SAD) mechanism after the C2f layer of the Neck module. This enhances the image features of small tissue regions in oral images (such as low-density radiolucent areas, high-density attachments, and areas with blurred boundaries); suppresses background interference from alveolar bone, mucosa, and soft tissue; maintains extremely low parameter and computational loads to meet real-time inference requirements; and outputs enhanced multi-scale fusion features, providing high-quality input for the small target region recognition head, thereby improving the accuracy of subsequent quantization feature parameter extraction.

[0034] like Figure 2 , Figure 3 As shown, the basic YOLOv8 model includes a Backbone module, a Neck module, and a Head module. The Backbone module includes cascaded Con-C2f-1, Con-C2f-2, and SPPF-C2f layers. The Neck module includes convolutional sampling units, a first-scale unit, a second-scale unit, and a third-scale unit. The Head module includes parallel Conv-9, Conv-10, and Conv-11 layers.

[0035] The convolutional sampling unit includes a cascaded Upsample-1 layer, a Concat-2 layer, a C2f-5 layer, and a Conv-6 layer. The first scale unit includes an Upsample-2 layer, a Concat-1 layer, a C2f-6 layer, a SAD-1 layer, and a Conv-7 layer. The second scale unit includes a Concat-3 layer, a C2f-7 layer, a SAD-2 layer, and a Conv-8 layer. The third scale unit includes a Concat-4 layer, a C2f-8 layer, and a SAD-3 layer.

[0036] The Con-C2f-1 layer comprises a series of Conv-1, C2f-1, Conv-2, and C2f-2 layers. The Con-C2f-2 layer comprises a series of Conv-3 and C2f-3 layers. The SPPF-C2f layer comprises a series of Conv-4, SPPF, C2f-4, and Conv-5 layers; the SPPF layer comprises a series of Conv-12, Maxpool units, Concat-5, and Conv-13 layers; the Maxpool units comprise parallel and structurally identical Maxpool-1, Maxpool-2, and Maxpool-3 layers.

[0037] In this embodiment, Conv-1 to Conv-13 layers have the same structure, all including a series of Conv2d layers, BN (batch normalization) layers, and SILU (activation function) layers.

[0038] The SAD-1 to SAD-3 layers have the same structure, such as Figure 4 As shown, all layers include a cascaded spatial attention layer, a channel attention layer, and a feature residual layer. Taking the SAD-1 layer as an example, the corresponding process is as follows:

[0039] S2-1. Perform feature grouping and channel shuffling on the output features of layer C2f-6 to generate a grouped shuffling feature map.

[0040] Specifically, the output features of layer C2f-6 are equally divided into G groups along the channel direction, with each group containing the same number of channels. Then, a channel shuffling operation is performed on the channels within each group, shuffling the channel order to enhance cross-channel information flow. Finally, based on the response correlation between channels and tooth positions (anterior teeth, posterior teeth, occlusal surfaces, and gingiva) learned autonomously during training, each group is adaptively reorganized into corresponding semantic groups, allowing feature channels from the same tooth position region to cluster together, facilitating collaborative enhancement by the subsequent attention module. The entire process is independent of external priors and is entirely completed by the model itself.

[0041] S2-2. Data augmentation is performed on the grouped shuffled feature maps through a spatial attention layer to generate spatially augmented feature maps;

[0042] Specifically, the spatial attention layer is used to enhance the spatial location response of the feature map to areas of small lesions. First, the input grouped and shuffled feature maps are subjected to global average pooling and global max pooling respectively to obtain two spatial descriptors. These two descriptors are then concatenated along the channel direction to form a dual-channel feature map containing average and max information. Subsequently, a 1×1 convolutional layer is used to reduce the dimensionality of the concatenated result, and a spatial weight map is generated by passing it through a sigmoid activation function. Each pixel value in this weight map represents the importance of the corresponding spatial location. Finally, the original grouped and shuffled feature map is multiplied element-wise with the spatial weight map, so that the model focuses on the teeth and lesion areas, while suppressing background noise such as alveolar bone and mucosa, and outputting a spatially enhanced feature map.

[0043] S2-3. Input the spatial augmentation feature map into the channel attention layer to generate a joint channel-spatial augmentation feature map;

[0044] Specifically, the channel attention layer is used to learn the contribution of different feature channels to lesion recognition. First, global average pooling is performed on the spatial augmentation feature map in the height and width dimensions, compressing each channel into a scalar to obtain a channel descriptor. Then, it is passed through two 1×1 convolutional layers in sequence: the first convolutional layer compresses the number of channels to 1 / r of the original number (r is the compression ratio) and processes it with the ReLU activation function; the second convolutional layer restores the original number of channels and generates a channel weight vector with each element corresponding to the importance weight of a channel through the Sigmoid activation function; finally, the spatial augmentation feature map and the channel weight vector are multiplied along the channel dimension to recalibrate the channels, enhance the response of sensitive channels such as lesion texture and edges, and output a channel-spatial joint augmentation feature map.

[0045] S2-4. Input the channel-space joint enhanced feature map and the output features of layer C2f-6 into the feature residual layer to generate the first SAD enhanced feature map.

[0046] Specifically, an inverse channel shuffling operation is performed on the channel-spatial joint enhanced feature map to restore the channel order of the feature map to its original arrangement before input. This inversely shuffled feature map is then element-wise added to the output features of layer C2f-6 to achieve residual connections. Residual connections ensure stable gradient propagation while preserving detailed information in the original features, avoiding information loss due to over-correction by the attention module. Finally, the first SAD enhanced feature map is output, with dimensions identical to the input, and can be directly fed into subsequent convolutional layers or the detection head. Furthermore, during inference, the computation of some attention branches can be dynamically skipped based on the confidence level of the candidate region, achieving dynamic lightweighting.

[0047] Existing general attention mechanisms (such as SE, CBAM, ECA, etc.) have significant limitations in the task of identifying small oral lesions. They are usually placed in the backbone module, which extracts global abstract features and lacks fine-grained control over multi-scale fusion information, making small lesion features easily submerged by the background. Their grouping strategy is fixed and uniform, which cannot adapt to the differences in feature distribution of lesions at different tooth positions (anterior / posterior teeth) and different scales in the oral cavity. Channel interaction only uses simple channel shuffling or excitation, lacking spatial and channel collaborative gating, making it difficult to suppress interference from complex backgrounds such as alveolar bone and gingiva. The computational cost is fixed and cannot be dynamically adjusted according to the difficulty of lesion identification, resulting in wasted computational resources. To address this, this embodiment embeds the ShuffleAttention mechanism into the bottom-up path of the Neck module. Since the Neck module has already completed top-down feature pyramid fusion, the bottom-up path can backfeed deep semantic information to shallow detailed features. Applying attention here can more accurately enhance the feature response of the small lesion area, while effectively suppressing background interference from alveolar bone, mucosa, etc., without affecting the basic feature extraction efficiency of the backbone network.

[0048] The processing steps of the model training module include:

[0049] S3-1. Collect standard oral images and their labels for training, train multimodal oral data and input them into the basic YOLOv8 model to obtain training detection and segmentation information;

[0050] The processing steps of the basic YOLOv8 model include:

[0051] S3-1-1. Input the training standard oral cavity images into the Backbone module to generate large-scale feature maps, medium-scale feature maps, and small-scale feature maps.

[0052] The training standard oral image is input into the Con-C2f-1 layer, and then pre-sampled by the Conv-1 layer to reduce spatial resolution and expand channel dimensions. The image is then subjected to cross-stage local feature fusion by the C2f-1 layer to enhance the local receptive field. Further downsampling occurs by the Conv-2 layer, and finally, the C2f-2 layer performs multi-scale feature aggregation to generate a large-scale feature map rich in detail, which is beneficial for subsequent detection of small tissue regions. The large-scale feature map is then input into the Con-C2f-2 layer, and subsequently downsampled by the Conv-3 layer and recombined with deep features by the C2f-3 layer to generate a mesoscale feature map. This mesoscale feature map is then input into the SPPF-C2f layer, and channel adjustment is performed by the Conv-12 layer. Multi-scale contextual information is captured by a Maxpool unit containing three parallel max-pooling layers, and the pooling results are fused by the Concat-5 layer. The output data of Concat-5 and Conv-12 are merged, and then the fused features are output by Conv-13 to finally generate the small-scale feature map with the strongest semantic information, providing high-level semantic guidance for the cross-scale feature fusion of the neck network in the future.

[0053] S3-1-2. Input the large-scale feature map, medium-scale feature map and small-scale feature map into the Neck module to generate the first SAD enhanced feature map, the second SAD enhanced feature map and the third SAD enhanced feature map.

[0054] S3-1-2 includes:

[0055] A1. Input the small-scale feature map and the medium-scale feature map into the convolutional sampling unit to generate a downsampled convolutional feature map;

[0056] Specifically, the small-scale feature map is input into the Upsample-1 layer for upsampling by a factor of 2, thereby expanding the spatial size to match that of the mesoscale feature map. Figure 1 The upsampled feature map is then concatenated with the mesoscale feature map along the channel direction and fed into the Concat-2 layer to achieve the fusion of deep semantics and shallow details. Next, the C2f-5 layer performs cross-stage local feature fusion on the concatenated feature map to further refine multi-scale information. Finally, the Conv-6 layer downsamples the fused features through a convolution with a stride of 2, reducing the spatial size and increasing the number of channels, and outputs a downsampled convolutional feature map for use in the subsequent bottom-up PAN path. The convolutional sampling unit, through its design of upsampling followed by downsampling, effectively promotes the transfer of deep semantics to shallow features and enhances the model's ability to perceive multi-scale targets.

[0057] A2. Input the large-scale feature map and the downsampled convolutional feature map into the first scale unit to generate the first SAD enhanced feature map and the first fused convolutional feature map;

[0058] Specifically, the downsampled convolutional feature map is input into the Upsample-2 layer for upsampling by a factor of 2, so that the spatial size is restored to the same level as the large-scale feature map. Figure 1 The Concat-1 layer concatenates the upsampled feature map with the large-scale feature map along the channel direction, achieving the fusion of deep semantics and shallow high-resolution details. The C2f-6 layer performs cross-stage local feature fusion on the concatenated feature map to further refine multi-scale information. Then, the SAD-1 layer adaptively groups, shuffles channels, and performs spatial and channel-gated enhancement on the fused features output from the C2f-6 layer to suppress background interference and highlight the feature responses of micro-tissue regions, generating the first SAD-enhanced feature map. Finally, the Conv-7 layer adjusts the number of channels in the fused features output from the C2f-6 layer through convolution, generating the first fused convolutional feature map. The first scale unit, by introducing the SAD attention mechanism, significantly improves the detection sensitivity of micro-tissue regions such as early demineralization areas and plaque attachment areas.

[0059] A3. Input the first fused convolutional feature map and the downsampled convolutional feature map into the second scale unit to generate the second SAD enhanced feature map and the second fused convolutional feature map;

[0060] Specifically, the Concat-3 layer concatenates the first fused convolutional feature map and the downsampled convolutional feature map along the channel direction, fusing deep semantics from the top-down path with detailed information from the bottom-up path. Then, the C2f-7 layer performs cross-stage local feature fusion on the concatenated feature map to further refine multi-scale features. The SAD-2 layer adaptively groups, channels shuffles, and performs dual spatial and channel gating enhancement on the fused features output by the C2f-7 layer to generate a second SAD enhanced feature map, effectively suppressing background noise and enhancing the feature response of the medium-scale lesion region. Finally, the Conv-8 layer downsamples the fused features output by the C2f-7 layer through a convolution with a stride of 2 to generate a second fused convolutional feature map. The second scale unit improves the localization and feature extraction accuracy of medium-scale lesion regions (such as caries in the molar region) through the synergistic effect of the attention mechanism and cross-scale fusion.

[0061] A4. Input the second fused convolutional feature map and the small-scale feature map into the third scale unit to generate the third SAD enhanced feature map.

[0062] Specifically, the Concat-4 layer concatenates the small-scale feature map with the downsampled feature map output by the Conv-8 layer along the channel direction, enabling deep fusion of the semantic information of the highest layer with the mid-layer features passed from the bottom up. Then, the C2f-8 layer performs cross-stage local feature fusion on the concatenated feature map to further extract multi-scale contextual features. Finally, the SAD-3 layer (which adaptively groups, shuffles, and performs dual spatial and channel gating enhancement on the fused features, further suppresses background interference and enhances the feature response of large-scale lesion areas (such as low-density areas around the apex) on the feature map with the richest semantics, outputting the third SAD enhanced feature map. The third-scale unit effectively improves the feature representation ability of large-area or deep lesions by introducing an attention mechanism at a high semantic layer.

[0063] S3-1-3. Input the first SAD enhanced feature map, the second SAD enhanced feature map, and the third SAD enhanced feature map into the Head module to generate the corresponding training detection segmentation information.

[0064] Specifically, the Head module employs three parallel detection heads, receiving the first, second, and third SAD enhanced feature maps, respectively. Each detection head first performs channel adjustment and nonlinear transformation on the input feature map through a corresponding convolutional layer (Conv-9, Conv-10, or Conv-11). Then, it performs target classification, bounding box regression, and pixel-level segmentation prediction: the classification branch outputs the probability that the candidate region belongs to the lesion or background, the regression branch outputs the bounding box coordinates, and the segmentation branch generates a binary segmentation mask with the same resolution as the input image through upsampling and pixel-by-pixel classification. Simultaneously, each detection head extracts the feature map from the previous layer as a 256-dimensional deep visual feature vector. The three detection results are summarized by scale, outputting the bounding box, tooth position number (mapped to the tooth position through post-processing), confidence score, pixel-level segmentation mask, and deep visual feature vector for each training candidate region. This constitutes the training detection and segmentation information, which is used by the subsequent cross-modal fusion module.

[0065] S3-2. Based on the training multimodal oral data and the segmentation information and labels of each training detection, determine the individual standardization factor and calculate the corresponding candidate region density, tooth position basic dynamic weight, difficult sample basic dynamic weight and marginal basic dynamic weight; the individual standardization factor includes age standardization coefficient, tooth volume-bone volume mismatch index and anterior and posterior tooth partition coefficient.

[0066] In this embodiment, the training multimodal oral data includes age. Total number of teeth 1. Dental arch width / height (extracted from CBCT images or panoramic films) 2. Tooth position number (11-48, distinguishing between anterior and posterior teeth).

[0067] Based on the pixel-level segmentation mask in the training detection segmentation information of each training candidate region, the proportion of foreground pixels in the segmentation mask to the total pixels within the bounding box is calculated to obtain the corresponding candidate region density. .

[0068] To eliminate the impact of differences in oral anatomy among patients of different ages on the training of the basic YOLOv8 model: Children (6-12 years old) are in the primary or mixed dentition stage, with unstable tooth positions and smaller tooth sizes, while adults (18 years and older) have fixed tooth positions and more significant crowding. Using uniform weights would lead to the model being too strict in detecting children's regions or insufficient in penalizing crowding in adults' regions. Therefore, age-standardized coefficients were set based on clinical oral anatomy experience, with age-standardized coefficients for children, adolescents, and adults. The values ​​were 0.85, 1.0, and 1.15, respectively. The age-standardized coefficient was used to calculate the overall tooth crowding, which reduced the penalty intensity in the children's region due to developmental immaturity, while increasing the focus on crowded and small areas in the adult region. This balanced the detection performance of the model across different age groups, ultimately improving the accuracy and robustness of oral tissue region feature extraction across all age groups.

[0069] Based on training multimodal oral cavity data, the width of a single tooth and the total length of the dental arch are extracted from the images. Obtain the total amount of teeth (The sum of the widths of all teeth). Based on the total length of the dental arch and the total number of teeth, according to the formula:

[0070] ;

[0071] Calculate the tooth volume-bone volume mismatch index In this embodiment, when A value greater than 1.1 indicates that the patient's teeth are crowded; when A value less than 0.9 indicates that the patient has a sparse dental arch.

[0072] Based on oral anatomy, the anterior teeth (numbered 11-13, 21-23, 31-33, 41-43) have lower crowding and typically larger training candidate regions, while the posterior teeth (numbered 14-18, 24-28, 34-38, 44-48) have high crowding, numerous micro-tissue areas, and are more difficult to identify. To differentiate the contributions of the anterior and posterior teeth in the loss function, the anterior and posterior tooth partitioning coefficients for the anterior teeth are used. A value of 0.7 reduces the weight of this region in the loss calculation because it is relatively easy to identify. Anterior and posterior tooth partitioning coefficients for the posterior tooth region. A value of 1.3 increases the weight of this region on the loss, forcing the model to focus on the difficult region.

[0073] Given any training candidate region, based on the ground truth bounding box coordinates and image size of the corresponding label, combined with the age-normalized coefficient and the total length of the dental arch, according to the formula:

[0074] ;

[0075] ;

[0076] ;

[0077] Calculate the scale ratio corresponding to the training candidate region. .in, , These represent the horizontal and vertical coordinates of the top-left corner of the actual bounding box, respectively. , These represent the horizontal and vertical coordinates of the bottom right corner of the actual bounding box, respectively. Represents the actual area of ​​the bounding box. Represents the standardized image area of ​​an individual. , These represent the height and width of the image, respectively.

[0078] Finally, based on the proportion of lesion size, dynamic weights were set for tooth position, difficult samples, and marginal samples. A value less than 0.05 indicates that the candidate training region is a small tissue region, and the tooth position-based dynamic weights are then adjusted accordingly. Difficult sample basic dynamic weights and edge-based dynamic weights The values ​​are 0.6, 1.4, and 1.6 respectively; when If the range is [0.05, 0.15), it indicates that the training candidate region is a medium-sized tissue region, and the tooth position baseline dynamic weight is then applied. Difficult sample basic dynamic weights and edge-based dynamic weights The values ​​are 1.0, 1.0, and 1.0 respectively; when A value not less than 0.15 indicates that the candidate training region is a large tissue region, thus the tooth position-based dynamic weight is... Difficult sample basic dynamic weights and edge-based dynamic weights The values ​​are 1.4, 0.6 and 0.4 respectively.

[0079] S3-3. Based on the segmentation information and labels of each training detection, calculate the corresponding tooth crowding penalty term enhancement loss, hard sample focusing enhancement loss and edge enhancement loss;

[0080] Taking the training detection and segmentation information of any training candidate region as an example, S3-3 includes:

[0081] S3-3-1. Based on the training detection segmentation information and its labels, calculate the tooth crowding penalty term enhancement loss;

[0082] Specifically, based on the bounding boxes (predicted boxes) and labeled ground truth boxes of the training detection and segmentation information, taking any predicted box and its corresponding ground truth box as an example, according to the formula:

[0083] ;

[0084] ;

[0085] ;

[0086] Calculate the intersection area Union area and intersection .

[0087] According to the formula:

[0088] ;

[0089] ;

[0090] ;

[0091] ;

[0092] ;

[0093] Calculate the minimum bounding rectangle .

[0094] Then, according to the formula:

[0095] ;

[0096] ;

[0097] Calculate the penalty term for tooth crowding to enhance loss .

[0098] in, , These represent the maximum value function and the minimum value function, respectively. , These represent the x and y coordinates of the top left corner of the prediction box, respectively. , These represent the horizontal and vertical coordinates of the bottom right corner of the prediction box, respectively. , These represent the horizontal and vertical coordinates of the top-left corner of the true bounding box, respectively. , These represent the horizontal and vertical coordinates of the bottom right corner of the true bounding box, respectively. , These represent the areas of the predicted bounding box and its corresponding ground truth bounding box, respectively. , , , These represent the left, top, right, and bottom boundaries of the bounding rectangle, respectively. It represents a generalized intersection, union, and comparison.

[0099] S3-3-2, Calculate the focusing enhancement loss for difficult samples; the focusing enhancement loss for difficult samples adopts the standard FocalLoss function.

[0100] S3-3-3. Based on the training detection and segmentation information and its labels, determine the edge pixels and non-edge pixels of the corresponding training candidate regions, and calculate the edge enhancement loss.

[0101] Specifically, based on the predicted segmentation mask and the true segmentation mask with its labels, which are obtained from the true segmentation mask, edge masks are obtained through Canny edge detection or morphological dilation difference (dilation map minus the original image). The edge mask and the true segmentation mask have the same size. In the edge mask, a pixel value of 1 represents an edge pixel, and a pixel value of 0 represents a non-edge pixel.

[0102] Based on the pixel positions of each pixel in the edge mask, weight values ​​are defined for edge pixels and center pixels, and a corresponding edge weight map is constructed. In this embodiment, the weight of the edge pixels is twice that of the center pixels.

[0103] Then, according to the formula:

[0104] ;

[0105] ;

[0106] ;

[0107] Calculate the weighted true instance pixels and (Pixels that are correctly predicted and are foreground pixels are weighted by edge / center), the sum of the weighted predicted foreground pixels. Weighted sum of true foreground pixels .in, This represents the summation function. Indicates the pixel position as The weight, Indicates the pixel position as Predicted segmentation mask, Indicates the pixel position as The true segmentation mask.

[0108] Then, according to the formula:

[0109] ;

[0110] ;

[0111] Calculate edge enhancement loss .in, This represents the edge perception coefficient.

[0112] S3-4. Based on individual standardized factors, optimize the basic dynamic weights of each tooth position, each difficult sample, and each marginal sample to generate the corresponding dynamic weights of each tooth position, difficult sample, and marginal sample.

[0113] It should be explained that although the dynamic weights based on tooth position, difficult samples, and edges can initially balance the loss contribution of regions of different sizes, they ignore the more complex anatomical prior factors in oral imaging. For example, the inherent high crowding in the posterior tooth region can lead to bounding box overlap and localization offset, and scale weights alone cannot enhance the localization constraint on dense regions. Low-confidence difficult samples (such as early low-density radiolucent areas) require a stronger focusing mechanism, but scale weights treat easy and difficult samples equally, which can easily cause the model to fall into over-optimization of easy samples. In addition, regions of different densities (such as sparse attachments and dense calcification areas) have significantly different requirements for segmentation edge accuracy, and scale weights cannot adaptively adjust the edge loss. Therefore, this embodiment optimizes the three basic weights through individual normalization factors, so that the tooth position crowding penalty term enhancement loss, difficult sample focusing enhancement loss, and edge enhancement loss can adjust the optimization intensity in real time according to the individual attributes of each candidate region, thereby solving the problems of inaccurate localization, insufficient learning of difficult samples, and blurred segmentation edges caused by basic weights.

[0114] Taking any training candidate region as an example, S3-4 includes:

[0115] S3-4-1. Calculate the adjacent intersection-union ratio (IU) of the bounding boxes of the training candidate regions and their adjacent bounding boxes, and filter them. Calculate the average IU of the corresponding adjacent intersection-union ratios.

[0116] Specifically, the intersection-union ratio (IUR) of each bounding box and its adjacent bounding boxes is calculated. The IURs are sorted in descending order, and the top three IURs are averaged to obtain the average IUR.

[0117] S3-4-2. Based on the age-standardized coefficient, tooth-bone imbalance index, and anterior and posterior tooth zoning coefficient, optimize the basic dynamic weight of tooth position to generate dynamic weight of tooth position.

[0118] The formula corresponding to S3-4-2 is:

[0119] ;

[0120] ;

[0121] ;

[0122] ;

[0123] in, This represents the density factor of a single tooth image. Indicates the overall degree of tooth crowding. This represents the average of adjacent intersection-union ratios. Indicates the dynamic weight of tooth position. This represents the congestion penalty coefficient. This represents the total number of teeth detected in the training candidate region. Represents the age-standardized coefficient. Indicates the tooth-bone mismatch index. Indicates the zoning coefficient for anterior and posterior teeth. This represents the basic dynamic weight of tooth position.

[0124] S3-4-3. Based on the age standardization coefficient and the anterior and posterior tooth partition coefficient, and combined with the confidence in the training detection segmentation information, the basic dynamic weights of difficult samples are optimized to generate dynamic weights for difficult samples.

[0125] Specifically, the basic weights are only coarse-grained based on region scale (large / medium / small), which fails to reflect the significant differences in recognition difficulty among different samples at the same scale. For example, early low-density radiolucent areas of the same size are more difficult to identify in children with unstable tooth positions during the mixed dentition period than in adults, and are more easily interfered with by background in crowded posterior tooth areas than in anterior tooth areas. Therefore, this embodiment constructs a difficult sample intensity factor based on age-standardized coefficients and anterior / posterior tooth partitioning coefficients, combined with the confidence score output by the model (the lower the confidence score, the more difficult the sample). For example, when the sample is located in the posterior tooth area (… =1.3) or the patient is a child ( When the value is 0.85, the dynamic weight of hard samples is automatically amplified, so that the low confidence region receives a higher loss weight.

[0126] The dynamic weights for difficult samples generated in this embodiment allow the model to no longer treat all samples equally during training. Instead, it allocates its limited representational capabilities to areas that are difficult to identify due to complex anatomical structures or special developmental stages, effectively alleviating the problem of missed detection in vague and small tissue areas such as early demineralization areas and dental plaque attachment areas.

[0127] Therefore, the formula corresponding to S3-4-3 is:

[0128] ;

[0129] ;

[0130] ;

[0131] in, Indicates the intensity factor of difficult samples. This indicates the confidence level of the training candidate region. Indicates the focus enhancement coefficient. This represents the dynamic weights of difficult samples.

[0132] S3-4-4. Based on the candidate region density and the anterior and posterior tooth partition coefficients, the edge-based dynamic weights are optimized to generate edge dynamic weights.

[0133] The formula corresponding to S3-4-4 is:

[0134] ;

[0135] ;

[0136] in, Indicates density weight, This represents the dynamic weights at the edges.

[0137] S3-5. Based on the dynamic weights of each tooth position, each difficult sample, and each edge, the corresponding tooth position crowding penalty term enhancement loss, difficult sample focusing enhancement loss, and edge enhancement loss are weighted and fused to generate an oral composite loss function.

[0138] Taking any training candidate region as an example, according to the formula:

[0139] ;

[0140] Calculate the oral cavity loss function for the training candidate region. .in, This represents the hard sample focusing enhancement loss for the training candidate region.

[0141] The oral loss functions of all training candidate regions are integrated, and the average value of all oral loss functions is calculated and used as the oral composite loss function.

[0142] S3-6. Based on the oral composite loss function, the basic YOLOv8 model is trained using the AdamW optimizer to generate an improved YOLOv8 model.

[0143] like Figure 5As shown, the training loss of the improved YOLOv8 model in this embodiment rapidly decreases and converges to near zero, while the validation loss continues to decrease to 0.036, indicating stable model convergence and no overfitting. The top-5 accuracy remains consistently 1.000. Overall evaluation shows that this invention achieves excellent performance in oral micro-tissue region detection and feature quantization tasks by embedding the SAD attention mechanism, incorporating oral anatomy priors into the oral composite loss function, and cross-modal fusion, achieving high precision, high recall, and high robustness. Figure 5 In this context, "result" refers to the true, unprocessed metric values ​​(such as loss or accuracy) recorded by the improved YOLOv8 model after each round or step of training, while "smooth" is a curve processed by applying an exponential moving average (EMA) or other smoothing algorithm to the original result data.

[0144] Therefore, a lightweight ShuffleAttention mechanism is embedded in the neck network of the YOLOv8 model, enabling the model to focus more effectively on key lesion areas in oral images with lower computational cost, suppressing background noise interference and thus enhancing the perception of small lesion features. Simultaneously, for these small lesions, an oral composite loss function is adopted, where the loss weights are no longer manually preset constants but dynamic coefficients calculated in real-time by the model based on the lesion's own attributes during forward inference; backpropagation updates and improves all parameters of YOLOv8; the attention module, recognition head, and segmentation branch are optimized; and the model's recall rate, localization accuracy, and segmentation edge quality for small lesions are improved.

[0145] When extracting and quantifying features from oral lesion imaging, single-modal data (images or text only) is insufficient to fully describe the morphology and clinical context of the tissue region: images can provide visual features such as low-density areas, boundaries, and textures, but lack the perception of semantic information such as patient symptoms and medical history; textual medical records document key clues such as tooth position symptoms, patient age, and previous treatments, but cannot be precisely aligned with the spatial location in the images. To overcome this limitation, this embodiment introduces a cross-modal attention fusion module, which performs bidirectional interaction and alignment of structured text data, initial features of patient background text, and detection and segmentation information of each candidate region in the same semantic space, focusing on the corresponding tooth position and lesion region. This provides richer and more accurate feature representations for subsequent regression quantification of feature parameters, thereby effectively improving the accuracy and robustness of oral tissue region feature extraction. The model training module in this embodiment is mainly used for offline training.

[0146] In the cross-modal fusion module, the detection and segmentation information of each candidate region is fused with the structured text data and the initial features of the patient background text. Taking the detection and segmentation information of any candidate region as an example, the processing procedure of the cross-modal fusion module includes:

[0147] S5-1. Encode the detection and segmentation information of the candidate region to generate the corresponding visual embedding and prior embedding.

[0148] Specifically, the depth visual features of the candidate region are processed through a linear projection layer to generate a visual embedding. Then, the tooth position number, confidence score, and normalized bounding box coordinates from the detection and segmentation information of the candidate region are concatenated into an original prior vector, which is then input into a fully connected layer to generate a prior embedding.

[0149] S5-2. The structured text data, prior embeddings, and visual embeddings are processed through a bidirectional cross-attention mechanism to generate visual-text fusion attention features.

[0150] Specifically, using structured text data as the query vector and visual embeddings as the key and value vectors, the attention output of text-guided vision is calculated to obtain text-visual attention features. This allows symptom keywords in medical records to focus on the corresponding tissue regions of teeth in the images. Then, using visual embeddings as query vectors and structured text data as key and value vectors, the attention output of the visually guided text is calculated, yielding visual-text attention features. This enhances the semantic weight of relevant descriptions in the text by incorporating features such as low-density areas and blurred edges in the image. The prior embedding is processed using a fully connected layer and a sigmoid activation function to obtain the gating factor. Then, the text-visual attention features and visual-text attention features are weighted and fused according to the gating factor, i.e., according to the formula:

[0151] ;

[0152] Obtain visual-text fusion attention features .

[0153] S5-3. The visual text fusion attention features and the initial features of the patient background text are fused and normalized to generate multimodal oral lesion image features of the candidate region.

[0154] Specifically, the visual text fusion attention features and the initial features of the patient background text are spliced ​​together, and then the spliced ​​data is normalized to generate multimodal oral lesion image features of the candidate region.

[0155] Repeat steps S5-1 to S5-3 until the multimodal oral lesion image features of each candidate region are calculated.

[0156] In this embodiment, the quantization model employs a fully connected model (three cascaded fully connected layers and an output layer) or a multimodal large model (BLIP-2 model and LoRA). Taking any candidate region as an example, the processing procedure of the fully connected model is as follows:

[0157] The multimodal oral lesion image features (1536 dimensions) of the candidate region are sequentially input into three fully connected layers (compressed to 512, 256, and 128 dimensions respectively), with each layer followed by a ReLU activation function to introduce non-linear expressive power. Then, the 128-dimensional features are mapped into a 5-dimensional continuous vector through an output layer (linear transformation), corresponding to lesion area, mean gray level, standard deviation of gray level, texture contrast, and edge gradient magnitude, respectively. Finally, numerical constraints are applied to the 5-dimensional continuous vector (e.g., the area is taken as non-negative, and the mean gray level is cropped to a reasonable range) to obtain the oral lesion image feature parameters of the candidate region. The fully connected model has a small number of parameters (typically only around two million), fast inference speed (milliseconds), and can stably converge on labeled data without relying on large-scale pre-trained models, making it easy to deploy lightweight on terminal devices and meet the needs of real-time auxiliary analysis in oral clinical practice.

[0158] The BLIP-2 model (Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models 2) was used as the quantization regression model, and LoRA (low-rank adaptation) was employed for parameter-efficient fine-tuning in the oral domain. The processing procedure for the multimodal large model is as follows:

[0159] The multimodal oral lesion image features output from step 5 are used as input, along with pre-defined oral professional text prompts (e.g., "Please output the following quantized feature parameters: lesion area (mm²), mean gray level (HU), gray level standard deviation, texture contrast, edge gradient magnitude"), and fed into the query transformer Q-Former of the BLIP-2 model. Q-Former utilizes learnable query embeddings and extracts the task features most relevant to the quantized parameters from the fused features through a cross-attention mechanism, generating 32 query embedding vectors. Then, all query embedding vectors are linearly projected and used as soft visual cues, inputting them into the frozen Large Language Model (LLM). During this process, LoRA low-rank adaptation is employed, injecting trainable low-rank matrices only into the self-attention layer of Q-Former and the query / key projection matrix in the LLM model, while keeping the main model parameters frozen. Finally, the LLM model outputs structured natural language text, such as "lesion area: 3.24 mm²; mean gray value: 85.6 HU; standard deviation of gray value: 12.3; texture contrast: 0.42; edge gradient magnitude: 0.78". After regular parsing the structured natural language text, the feature parameters of the oral lesion image are obtained.

[0160] Compared to fully connected models, multimodal large models, while increasing computational complexity and deployment resource requirements, can fully leverage the rich semantic priors gained during pre-training. Even with limited training samples, they can accurately understand the deep correlation between oral terminology and image features, thereby improving the feature quantification accuracy for blurred, small tissue regions such as early low-density areas and sparse attachments. LoRA fine-tuning updates only a very small number of parameters (typically 1%–5% of the total model parameters), effectively avoiding catastrophic forgetting and preserving the model's generalization ability on common image and language tasks. Furthermore, the output of multimodal large models is readable natural language text, facilitating intuitive verification and achieving higher robustness and accuracy.

[0161] In summary, this invention achieves precise localization and pixel-level segmentation of minute tissue regions (such as low-density radiolucent areas and high-density attachments) in oral images by embedding a SAD layer in the bottom-up PAN path of the YOLOv8 model, effectively improving recall and segmentation accuracy. It constructs an individualized oral composite loss function, incorporating oral anatomy priors (age, tooth and bone volume, tooth crowding, region density, etc.), dynamically adjusting the weights for localization, classification, and segmentation, enhancing the model's robustness in feature extraction for different patients, tooth positions, and tissue regions of different scales. Through a lesion prior-guided bidirectional cross-modal attention mechanism, it deeply integrates image visual features with medical record text semantics, suppressing background noise interference and solving the problem of fragmented multi-source information. Employing either a fully connected regression head or a multimodal large model with LoRA as optional quantization modules, it directly outputs objective continuous numerical parameters such as lesion area, grayscale mean, and texture contrast, achieving end-to-end automated feature extraction and quantization. This method features fast inference speed, small parameter count, and can be deployed on terminal devices, significantly improving the efficiency and quantization accuracy of oral image analysis.

[0162] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0163] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A system for quantifying oral lesion features based on an improved YOLOv8 model, characterized in that, include: The data acquisition and processing module is used to acquire multimodal oral data and perform preprocessing to generate standard oral images, structured text data, and initial features of patient background text. The model building module is used to build a basic YOLOv8 model based on the YOLOv8 model and the SAD attention mechanism. The model training module is used to train the basic YOLOv8 model, calculate the tooth crowding penalty term enhancement loss, hard sample focusing enhancement loss and edge enhancement loss, and generate an improved YOLOv8 model. The image detection and segmentation module is used to detect and segment standard oral images using an improved YOLOv8 model, generating detection and segmentation information for each candidate region. The detection and segmentation information includes bounding boxes, tooth position numbers, confidence scores, pixel-level segmentation masks, and depth visual features. The cross-modal fusion module is used to dynamically fuse structured text data, initial features of patient background text, and detection and segmentation information of each candidate region through a bidirectional cross-attention mechanism to generate multimodal oral lesion image features of each candidate region. The quantization module processes the multimodal oral lesion image features of each candidate region through a quantization model to generate corresponding oral lesion image feature parameters. The oral lesion image feature parameters include lesion area, mean gray level, standard deviation of gray level, texture contrast, and edge gradient amplitude.

2. The oral lesion feature quantification system based on the improved YOLOv8 model according to claim 1, characterized in that, The data acquisition and processing module includes the following processes: Collect multimodal oral cavity data; the multimodal oral cavity data includes image data and text data; The image data is denoised, size normalized, grayscale standardized, and contrast enhanced to generate image processing data. Semantic extraction, entity normalization, and redundancy removal are performed on the text data to generate structured text data and initial features of patient background text.

3. The oral lesion feature quantification system based on the improved YOLOv8 model according to claim 1, characterized in that, The basic YOLOv8 model includes a Backbone module, a Neck module, and a Head module; the Backbone module includes a cascaded Con-C2f-1 layer, a Con-C2f-2 layer, and an SPPF-C2f layer; the Neck module includes a convolutional sampling unit, a first scale unit, a second scale unit, and a third scale unit; The convolutional sampling unit includes a cascaded Upsample-1 layer, Concat-2 layer, C2f-5 layer, and Conv-6 layer; the first scale unit includes an Upsample-2 layer, Concat-1 layer, C2f-6 layer, SAD-1 layer, and Conv-7 layer; the second scale unit includes a Concat-3 layer, C2f-7 layer, SAD-2 layer, and Conv-8 layer; and the third scale unit includes a Concat-4 layer, C2f-8 layer, and SAD-3 layer.

4. The oral lesion feature quantification system based on the improved YOLOv8 model according to claim 3, characterized in that, The Con-C2f-1 layer includes a series of Conv-1, C2f-1, Conv-2, and C2f-2 layers; the Con-C2f-2 layer includes a series of Conv-3 and C2f-3 layers; the SPPF-C2f layer includes a series of Conv-4, SPPF, C2f-4, and Conv-5 layers; the SPPF layer includes a series of Conv-12, Maxpool cells, Concat-5, and Conv-13 layers.

5. The oral lesion feature quantification system based on the improved YOLOv8 model according to claim 3, characterized in that, The processing procedure of the SAD-1 layer includes: The output features of layer C2f-6 are grouped and shuffled to generate grouped and shuffled feature maps. Data augmentation of the grouped shuffled feature maps is performed through a spatial attention layer to generate spatially augmented feature maps; The spatial augmented feature map is input into the channel attention layer to generate a joint channel-spatial augmented feature map. The channel-spatial joint enhanced feature map and the output features of the C2f-6 layer are input into the feature residual layer to generate the first SAD enhanced feature map.

6. The oral lesion feature quantification system based on the improved YOLOv8 model according to claim 1, characterized in that, The processing steps of the model training module include: Collect standard oral images and their labels for training, train multimodal oral data, and input them into the base YOLOv8 model to obtain training detection and segmentation information; Based on the training multimodal oral data and the segmentation information and labels of each training test, the individual standardization factor is determined, and the corresponding candidate region density, tooth position basic dynamic weight, difficult sample basic dynamic weight and edge basic dynamic weight are calculated. Based on the segmentation information of each training detection and its label, the corresponding tooth crowding penalty term enhancement loss, hard sample focusing enhancement loss and edge enhancement loss are calculated; Based on individual standardized factors, the basic dynamic weights of each tooth position, each difficult sample, and each marginal sample are optimized to generate the corresponding dynamic weights of each tooth position, the dynamic weights of each difficult sample, and the dynamic weights of each marginal sample. Based on the dynamic weights of each tooth position, each difficult sample, and each edge, the corresponding tooth position crowding penalty term enhancement loss, difficult sample focusing enhancement loss, and edge enhancement loss are weighted and fused to generate an oral composite loss function. Based on the oral composite loss function, the basic YOLOv8 model is trained using the AdamW optimizer to generate an improved YOLOv8 model.

7. The oral lesion feature quantification system based on the improved YOLOv8 model according to claim 6, characterized in that, The individual standardized factors include the age standardized coefficient, the tooth-bone mismatch index, and the anterior and posterior tooth partition coefficient.

8. The oral lesion feature quantification system based on the improved YOLOv8 model according to claim 7, characterized in that, The generation of corresponding dynamic weights for tooth positions, dynamic weights for difficult samples, and dynamic weights for edge samples includes: Calculate the boundary boxes of the training candidate regions and the adjacent boundary boxes' intersection-union ratios, and calculate the average of the corresponding adjacent intersection-union ratios; Based on the age-standardized coefficient, tooth-bone mismatch index, and anterior and posterior tooth zoning coefficient, the basic dynamic weight of tooth position is optimized to generate dynamic weight of tooth position. Based on the age-standardized coefficient and the anterior and posterior tooth partition coefficient, and combined with the confidence in the training detection segmentation information, the basic dynamic weights of hard samples are optimized to generate dynamic weights for hard samples. Based on the candidate region density and the anterior and posterior tooth partition coefficients, the edge-based dynamic weights are optimized to generate edge dynamic weights.

9. The oral lesion feature quantification system based on the improved YOLOv8 model according to claim 1, characterized in that, The processing procedure of the cross-modal fusion module includes: The detection and segmentation information of any candidate region is feature-encoded to generate the corresponding visual embedding and prior embedding. A bidirectional cross-attention mechanism is used to process structured text data, prior embeddings, and visual embeddings to generate visual-text fusion attention features. The visual text fusion attention features and the initial features of the patient background text are fused and normalized to generate multimodal oral lesion image features of the candidate region.

10. The oral lesion feature quantification system based on the improved YOLOv8 model according to claim 1, characterized in that, The quantization model adopts either a fully connected model or a multimodal large model.

Images