Oral disease detection method based on large artificial intelligence model

Abstract

The present invention relates to the technical field of stomatology. Disclosed is an oral disease detection method based on a large artificial intelligence model. The method comprises: acquiring a dental lesion image and a corresponding intraoral state description; using a feature extraction module BERT and a multi-scale feature extraction module (MFEM) to respectively extract text feature information of an intraoral state and image feature information of the dental lesion image; identifying and excluding false negative samples therein; introducing a total loss function and a dynamic label assignment strategy into a YOLOv8 network to obtain an improved YOLOv8 network; dividing a data set into a training set and a test set, and sequentially inputting the training set and the test set into the improved YOLOv8 network for validation, so as to obtain a validated YOLOv8 network; and inputting the dental lesion image and the corresponding text information into the validated YOLOv8 network, so as to obtain lesion region type detection and a text description. The present invention can effectively improve the detection and classification of various intraoral diseases, and can give corresponding lesion region descriptions and suggestions.

Description

An oral disease detection method based on an artificial intelligence large model TECHNICAL FIELD

[0001] The present application relates to the technical field of oral medicine, and particularly relates to an oral disease detection method based on an artificial intelligence large model. BACKGROUND

[0002] With the improvement of living standards, people have higher requirements for oral health and gradually realize the importance of oral health. In fact, oral diseases not only cause local infection, tissue defect, etc., but also damage the overall health through affecting systemic diseases, nutritional intake, etc., and hinder physiological functions and facial aesthetics. If not treated in time, it will affect mastication, pronunciation and aesthetics after tooth loss, and seriously reduce the quality of life. In addition, if malignant diseases (such as oral cancer, etc.) are not diagnosed and treated in time, they may further develop and cause irreparable damage, seriously endangering the overall health. If infectious diseases are not controlled as soon as possible, they may spread to the whole body, interfering with the treatment of digestive system, endocrine, cardiovascular and other diseases. Therefore, early diagnosis of oral diseases is of great significance to maintaining oral health.

[0003] However, there is still a large gap in oral medical resources; there are many types of oral diseases (specific classifications include maxillofacial surgical diseases, dental pulp diseases, periodontal diseases, oral mucosa diseases, malocclusion, tooth loss, etc.), many of which have relatively hidden symptoms and require a high level of professional expertise for diagnosis. In addition, due to the difficulty in distinguishing the severity of oral diseases (I, II, III, etc.), patients themselves often cannot immediately realize the severity of oral diseases, which may lead to repeated registration or failure to seek medical treatment in time. In addition, due to the varying professional levels of oral doctors in different regions, young and primary doctors are more likely to miss or misdiagnose some diseases due to lack of experience, which may further lead to failure to diagnose and effectively treat diseases in time, delaying the best diagnosis and treatment time. Under the background of artificial intelligence, the medical industry is expected to use new tools for intelligent diagnosis or auxiliary treatment to provide strong support for doctors to make more accurate decisions. At present, the existing deep learning models such as convolutional neural network, generative adversarial network, recurrent neural network, autoencoder and deep belief network can realize the extraction of image features, automatic classification and identification of medical images, and qualitative and quantitative analysis of lesions. However, in the detection of oral diseases, due to the complexity of oral diseases (such as ulcers, dental caries, periodontitis, etc.), and the difficulty in distinguishing oral diseases of the same type and different degrees (such as recurrent oral ulcers and oral squamous cell carcinoma, which have similar lesion area colors and shapes, and need to be assisted by the surrounding area of the lesion tissue for diagnosis), etc., the existing intelligent diagnosis methods cannot effectively identify the types and grades of oral diseases.

[0004] Therefore, the present application provides an oral disease detection method based on an artificial intelligence large model to solve the problems existing in the prior art, which is a problem that needs to be solved by those skilled in the art. SUMMARY

[0005] Therefore, the present application provides an oral disease detection method based on an artificial intelligence large model to solve the problems existing in the prior art, which is a problem that needs to be solved by those skilled in the art.

[0006] To achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0007] An oral disease detection method based on an artificial intelligence large model, comprising the following steps:

[0008] Data acquisition step: acquiring tooth lesion pictures and corresponding oral internal state descriptions;

[0009] Feature extraction step: using feature extraction block BERT and multi-scale feature extraction block MFEM to extract picture feature information of the tooth lesion pictures and text feature information of the oral internal state, respectively, to obtain a label data set;

[0010] Error elimination step: identifying and excluding false negative samples in the label data to obtain a preprocessed label data set;

[0011] Network improvement step: introducing a total loss function and a dynamic label assignment strategy into the YOLOv8 network to obtain an improved YOLOv8 network;

[0012] Training and verification step: dividing the preprocessed label data set into a training set and a test set according to a ratio of 8:2, inputting the training set into the improved YOLOv8 network for training to obtain a trained YOLOv8 network, inputting the test set into the trained YOLOv8 network for verification to obtain a verified YOLOv8 network;

[0013] Disease monitoring step: inputting the tooth lesion pictures and the corresponding text information into the verified YOLOv8 network to obtain lesion region type detection and text description.

[0014] In the feature extraction step, the feature extraction block BERT uses a self-attention mechanism to calculate the influence of each word in a sentence on other words, and a mask language model MLM is used to pre-train the feature extraction block BERT.

[0015] Optionally, in the feature extraction step of the above method, the multi-scale feature extraction block (MFEM) includes: setting the input image to have four channels, inputting the four-channel output image into a 1x1 convolutional layer, a 3x3 convolutional layer, a 5x5 convolutional layer, and a max pooling layer respectively to obtain four corresponding tensors, concatenating the four tensors to form a new matrix tensor, fusing them through CotNet to generate a new feature information matrix and obtain the corresponding feature vector X, and then inputting the feature vector X into CotNet to obtain the output result.

[0016] Optionally, the CotNet method described above includes: converting the feature vector X into Queries: Q = XW. Q Keys: K = XW K Values: V = XW V Within a k×k spatial grid, k×k sets of convolutions are performed on all adjacent keys to obtain the local context information K between adjacent keys. 1 The local context information of the image is concatenated with Q to obtain the attention matrix D. The attention matrix D is then multiplied by V to obtain the complete context information K. 2 , with local context information K 1 The result is obtained by adding them together.

[0017] Optionally, in the above method, during the error removal step, the image features f generated by the cosine similarity calculation model are... i For text features f T The similarity between them is expressed as follows:

[0018] If Sim(f) i ,f T If the value of ) is higher than a certain threshold θ, and the sample is labeled as a negative sample in the labeled dataset, then it is labeled as a false negative sample.

[0019] In the above method, optionally, the total loss function expression in the network improvement step is: L total =θ cls L cls +θ loc L loc +θ scale L scale ,

[0020] Where, θ cls θ loc θ scale It is the weight hyperparameter, L cls For classification loss, L loc To locate the loss, L scale This is for scale loss.

[0021] Compared with the prior art, the oral disease detection method based on the artificial intelligence large model has the following beneficial effects: 1) the diagnosis performance of the system is effectively improved by the collection of the text features and the assistance of the prior knowledge, the identification and classification of the lesion types are performed, and the text description of the corresponding lesion is provided to assist the clinical doctors in decision-making; 2) the multi-scale feature extraction module MFEM is proposed to effectively enhance the information of the oral disease lesion area and further improve the algorithm diagnosis performance; 3) the detection and classification of various diseases in the oral cavity are effectively improved, and the corresponding lesion area description and suggestions are given. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of the provided drawings.

[0023] Fig. 1 is a flow chart of the oral disease detection method based on the artificial intelligence large model disclosed by the present application;

[0024] Fig. 2 is a principle diagram of the oral disease detection method based on the artificial intelligence large model disclosed by the present application;

[0025] Fig. 3 is a principle diagram of the multi-scale feature extraction module MFEM disclosed by the present application;

[0026] Fig. 4 is a principle diagram of CotNet disclosed by the present application. DETAILED DESCRIPTION

[0027] The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0028] In this application, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or equipment. Without more limitation, the element defined by the sentence "includes a" does not exclude the presence of other identical elements in the process, method, article or equipment including the element.

[0029] Referring to FIGS. 1 and 2, the present application discloses an oral disease detection method based on an artificial intelligence large model, comprising the following steps:

[0030] Data acquisition step: acquire tooth lesion pictures and corresponding oral internal state descriptions;

[0031] Feature extraction step: adopt feature extraction block BERT and multi-scale feature extraction block MFEM to extract picture feature information of tooth lesion pictures and text feature information of oral internal state respectively, and obtain label data set;

[0032] Error elimination step: identify and exclude false negative samples in the label data, and obtain preprocessed label data set;

[0033] Network improvement step: introduce total loss function and dynamic label assignment strategy in YOLOv8 network, and obtain improved YOLOv8 network;

[0034] Training and verification step: divide the preprocessed label data set into training set and test set according to the ratio of 8:2, input the training set into the improved YOLOv8 network for training to obtain the trained YOLOv8 network, input the test set into the trained YOLOv8 network for verification, and obtain the verified YOLOv8 network;

[0035] Disease monitoring step: input the tooth lesion pictures and corresponding text information into the verified YOLOv8 network, and obtain lesion region type detection and text description.

[0036] Further, in the data acquisition step, the tooth lesion pictures are patient oral cavity scanning data, and the number is greater than 2000 cases.

[0037] Further, in the feature extraction step, the feature extraction block BERT adopts self-attention mechanism to calculate the influence of each word on other words in the sentence, and adopts mask language model MLM to pretrain the feature extraction block BERT.

[0038] Specifically, BERT is a pre-training language model based on the Transformer architecture proposed by Google, which performs well in natural language processing (NLP) tasks and can capture the context information of words to achieve higher understanding ability. The core idea of BERT is to understand the context of a sentence in both directions (i.e., from left to right and from right to left), while traditional language models such as GPT only use single-direction context. This bidirectional context understanding makes BERT more accurate in capturing the relationship between words and context.

[0039] Further, the self-attention mechanism includes the representation X = [x1, x2, …, x n ] of a given input sequence, and the representation of each word is calculated by the following formula:

[0040] where Q (Query), K (Key), and V (Value) are representations obtained by linear transformation. d k is the dimension of Key. In BERT, the self-attention mechanism helps the model capture the dependency between all words in a sentence.

[0041] Further, during the pre-training process, some words in the input sequence are randomly masked, assuming that the masked word is w i , the goal of the model is to predict the original vocabulary of these words. The goal of the MLM task is to minimize the following loss function: L MLM = -∑ i∈M logP(w i |X),

[0042] where M is the set of masked words, and X is the input sequence after masking.

[0043] Further, referring to FIG. 3, in the feature extraction step, the multi-scale feature extraction block MFEM includes: letting the input picture channel number be four, the four-channel output picture is input into a 1x1 convolution layer, a 3x3 convolution layer, a 5x5 convolution layer, and a max pooling layer to obtain four corresponding tensors, the four tensors are spliced to form a new matrix tensor, they are fused by CotNet to generate a new feature information matrix and obtain a corresponding feature vector X, and then the feature vector X is input into the CotNet to obtain an output result.

[0044] Specifically, in most CNN architectures, researchers either stack the same size of convolutional layers as filters, or add a pooling layer, or use the image dual-channel parallel method to extract and fuse features. However, these methods only extract shallow features or deep features of one scale, and the extracted feature information is not comprehensive and cannot achieve the best recognition effect. Therefore, this patent proposes a Multi-scale Feature Extraction Module (MFEM) which extracts features of different scales in one layer of operation to obtain the depth features of the image and improve the recognition accuracy. The proposed feature extraction module uses four scales of convolution kernels to extract features of different scales.

[0045] Further, the input image channel is increased to four channels to increase the expression ability of the spectral image information. Different convolution sizes provide different receptive fields, which can extract features at different levels. The pooling operation itself has the effect of extracting features, and because there are no parameters, it will not produce overfitting. The 1x1 convolution layer, the 3x3 convolution layer, the 5x5 convolution layer and the maximum pooling layer each branch contains some convolution layers and batch normalization layers, and the ReLU activation function. Different convolution kernel size convolution layers extract different weight information to generate new effective features. The generated matrix with larger size can be first processed by dimension reduction, and the visual information is aggregated at different sizes, which is convenient for extracting features from different scales.

[0046] Further, referring to FIG. 4, CotNet includes: converting the feature vector X into Queries: Q=XW Q , Keys: K=XW K , Values: V=XW V , performing kxk group convolution on all adjacent keys in the kxk spatial grid to obtain local context information K 1 between adjacent keys, splicing the image local context information and Q to obtain an attention matrix D, and point multiplying the attention matrix D and V to obtain all context information K 2 . 1 Adding the local context information K

[0047] Further, W is a weight matrix, K 1 ∈R H×W×C , which can also be used as a static context representation of input X. Splicing processing obtains an attention matrix D through two consecutive 1x1 convolutions with and without ReLU activation function, respectively, and the expression is as follows: D=[K 1 ,Q]W θ W δ ,

[0048] where D represents the attention matrix, K 1 represents the image local context information, Q represents Queries, W θ and W δ represent two convolutions.

[0049] K 2 can also be used as dynamic context input, the expression is:

[0050] where V represents the value, the expression is: Y = K 1 + K 2 .

[0051] Different convolution layers are combined in parallel, and the result matrices processed by different convolution layers are spliced in the depth dimension to form a deeper matrix. In this way, the input image can be effectively processed, and more rich and diverse features can be extracted. It can efficiently expand the depth and width of the network, improve the accuracy of the deep learning network, and prevent overfitting

[0052] Further, in the error elimination step, the similarity between the image feature f i and the text feature f T is calculated by the cosine similarity calculation model, and the expression is as follows:

[0053] If the value of Sim(f i ,f T ) is higher than a certain threshold θ, and the sample is labeled as a negative sample in the label data set, it will be marked as a false negative sample.

[0054] Specifically, false negative samples refer to those data that are misclassified as negative samples (i.e., the model considers them irrelevant or mismatched), but in fact they should be positive samples (i.e., relevant or matched). The core idea of false negative exclusion FNE is to detect and eliminate these false negative samples through certain strategies, ensuring that the model only learns on true negative samples.

[0055] Further, if the similarity Sim(f i ,f T ) of the negative sample pair satisfies the following condition, it will be identified as a false negative sample: IF Sim(f i ,f T )>θ,then it is a False Negative,

[0056] where θ is a pre-set hyperparameter determined by experiment to balance the precision and recall rate of false negative sample detection.

[0057] Further, in the network improvement step, the total loss function expression is: L total = θ cls L cls + θ loc L loc + θ scale L scale ,

[0058] where θ cls , θ loc , θ scale are weight hyperparameters, L cls is the classification loss, L loc is the positioning loss, and L scale is the scale loss.

[0059] Further, the total loss function includes classification loss, positioning loss and scale loss.

[0060] Specifically, the classification loss: measures the gap between the predicted class and the true class, usually calculated using cross-entropy loss or Focal Loss, the expression is as follows:

[0061] where y i is the true class, and y z is the predicted class.

[0062] Positioning loss: measures the gap between the predicted box and the true box, usually calculated using IoU or GIoU, the expression is as follows: L loc = 1-IOU(A,B),

[0063] where A is the predicted box and B is the true box.

[0064] Scale loss: used to optimize the scale and aspect ratio of the predicted box to better fit the target.

[0065] Further, in the network improvement step, YOLOv8 introduces a dynamic label assignment strategy, which allows the model to more flexibly select positive and negative samples during training. This strategy adjusts the selection rules of positive and negative samples, which can better adapt to target detection tasks in complex scenarios.

[0066] The various embodiments described in this specification are described with reference to a particular sequence or order, but the order of the steps can be modified so that particular sequences or orders make no significant contribution to the progress of the art. Moreover, certain features and subcombinations are of utility and can be employed without reference to other features and subcombinations. For purposes of clarity, not every embodiment or feature in this specification is described or shown. Embodiments that provide real benefits can include any embodiment or combination of features described in this specification— even if the range of benefits realized is not the full range of benefits. Those of ordinary skill can understand that information and signals can be represented using any of a variety of technologies and techniques. For the purposes of this description, the terms "information" and "signals" can be regarded as synonymous. Those of ordinary skill can appreciate that the signals can be analog or digital, and the like.

[0067] The foregoing description of the disclosed embodiments enables one of ordinary skill in the art to make or utilize the application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other embodiments without the use of the inventive faculty. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for detecting oral diseases based on an artificial intelligence large model, characterized in that, The method comprises the following steps: Data acquisition step: acquire tooth lesion pictures and corresponding oral cavity internal state descriptions; Feature extraction step: The picture feature information of the tooth lesion pictures and the text feature information of the oral cavity internal state are extracted by using a feature extraction block BERT and a multi-scale feature extraction block MFEM, respectively, to obtain a label data set; Error elimination step: identify and exclude false negative samples in the label data to obtain a preprocessed label data set; Network improvement step: introduce a total loss function and a dynamic label assignment strategy into the YOLOv8 network to obtain an improved YOLOv8 network; Training and verification step: divide the preprocessed label data set into a training set and a test set according to a ratio of 8:2, input the training set into the improved YOLOv8 network for training to obtain a trained YOLOv8 network, input the test set into the trained YOLOv8 network for verification to obtain a verified YOLOv8 network; Disease monitoring step: input the tooth lesion pictures and the corresponding text information into the verified YOLOv8 network to obtain lesion region type detection and text description; In the feature extraction step, the multi-scale feature extraction block MFEM comprises: setting the input picture channel number to four, inputting a four-channel output picture into a 1x1 convolution layer, a 3x3 convolution layer, a 5x5 convolution layer and a maximum pooling layer to obtain four corresponding tensors, splicing the four tensors to form a new matrix tensor, fusing them through a CotNet to generate a new feature information matrix and obtain a corresponding feature vector X, and inputting the feature vector X into the CotNet to obtain an output result. In the error rejection step, the image feature f produced by the cosine similarity calculation model i is expressed by the following expression as the similarity between the text feature f T and the image feature f If the value of Sim(f i ,f T ) is higher than a certain threshold θ, and the sample is labeled as a negative sample in the label data set, it is marked as a false negative sample.

2. The oral disease detection method based on the artificial intelligence large model according to claim 1, wherein In the feature extraction step, the feature extraction block BERT adopts a self-attention mechanism to calculate the influence of each word on other words in a sentence, and adopts a mask language model MLM for pre-training of the feature extraction block BERT.

3. The oral disease detection method based on the artificial intelligence large model according to claim 1, wherein CotNet includes: converting the feature vector X into Queries: Q = XW Q , Keys: K = XW K , Values: V = XW V , obtaining local context information K between adjacent keys by k x k group convolution on all adjacent keys in a k x k space grid 1 , splicing the image local context information with Q to obtain attention matrix D, and point multiplying the attention matrix D with V to obtain all context information K 2 , adding the local context information K 1 to obtain the output result.

4. The oral disease detection method based on the artificial intelligence large model according to claim 1, wherein In the network improvement step, the total loss function expression is: L total = θ cls L cls + θ loc L loc + θ scale L scale , where θ cls , θ loc , and θ scale are weight hyperparameters, L cls is the classification loss, L loc is the localization loss, and L scale is the scale loss.

Images