Oral cavity image segmentation method and device based on SIFT algorithm, and medium

Abstract

The invention discloses an oral cavity image segmentation method and device based on an SIFT algorithm, and a medium. The method comprises the following steps: collecting an oral cavity image data set containing oral mucosa diseases; learning a scale invariant feature transform (SIFT) algorithm, and outputting a pixel-level SIFT descriptor vector about the oral cavity image data; the pixel-level SIFT descriptor vector is input into a Unet network, a corresponding segmentation result is output, and a Unet semantic segmentation model based on SIFT is constructed; and carrying out segmentation processing on specified oral cavity image data by utilizing a Unet semantic segmentation model based on SIFT (Scale Invariant Feature Transform). According to the method, the local feature descriptor of each pixel in the oral cavity image data is calculated by adopting the scale invariant feature transformation algorithm, and pixel-level segmentation of various oral cavity mucosa disease focus areas is realized by combining the Unet network, so that the segmentation precision of the oral cavity image can be improved, and the problem of oral cavity mucosa disease image focus segmentation is solved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an oral image segmentation method, device and medium based on SIFT algorithm. Background Art

[0002] Among oral diseases, oral mucosal diseases are common and complex to diagnose and treat. Oral potentially malignant disorders (OPMD) are a group of lesions that appear on the oral mucosa, including oral lichen planus (OLK), oral leukoplakia (OLP), and oral submucosal fibrosis (OSF). These lesions have the potential to develop into oral cancer. In recent years, due to the prevalence of unhealthy lifestyles, such as staying up late, excessive smoking and drinking, and unbalanced diet, the number of oral cancer patients among young people has increased significantly. Early diagnosis and treatment of OPMD are crucial to reducing the incidence of oral cancer. Studies have shown that about one-third of OPMD may transform into squamous cell carcinoma (SCC), so early identification, evaluation, and intervention are particularly important.

[0003] In the early identification and diagnosis of oral mucosal OPMD diseases, traditional manual identification has many limitations. First, the diagnosis results are highly dependent on the doctor's professional knowledge and clinical experience, and are easily affected by subjective factors, making it difficult to ensure the accuracy and consistency of the diagnosis. Secondly, the training cycle of high-level doctors is long and the cost is high, which makes it difficult to train in areas with scarce medical resources. With the rapid development of artificial intelligence technology, disease identification methods based on semantic segmentation have shown significant advantages. The semantic segmentation model uses deep learning technology to automatically extract and learn the characteristic information of the disease through training and learning of a large amount of medical imaging data, and quickly discover patterns or features that may be missed or take longer to be discovered by human clinicians, so as to achieve accurate identification and positioning of the disease. This method overcomes the subjectivity and uncertainty of manual identification and improves the objectivity and accuracy of diagnosis. In addition, the semantic segmentation model can also continuously improve the performance of disease identification and reduce the occurrence of misdiagnosis and missed diagnosis through continuous learning and optimization.

[0004] However, the current research on the application of deep learning methods to analyze oral mucosal OPMD disease images mainly focuses on disease classification and lesion area target detection. A few studies have classified and detected lesions for several oral mucosal diseases and oral cancer. However, these studies are mostly limited to target detection of a single disease, the detection classification is relatively simple, and the target detection output result is usually a set of rectangular frame coordinates and corresponding category labels, and the detection accuracy is very limited. Therefore, how to achieve high-precision lesion segmentation for oral mucosal diseases is a problem that technicians in this field need to solve. Summary of the invention

[0005] The embodiments of the present invention provide an oral image segmentation method, apparatus, computer equipment and storage medium based on the SIFT algorithm, aiming to improve the segmentation accuracy of oral images and solve the problem of lesion segmentation of oral mucosal disease images.

[0006] In a first aspect, an embodiment of the present invention provides an oral image segmentation method based on a SIFT algorithm, comprising: Obtain an oral image dataset containing oral mucosal diseases; Using the oral image data in the oral image data set to learn a scale-invariant feature transformation algorithm, and the scale-invariant feature transformation algorithm outputs a pixel-level SIFT descriptor vector about the oral image data; The pixel-level SIFT descriptor vector is input into the Unet network, and the Unet outputs the corresponding segmentation result, thereby constructing a SIFT-based Unet semantic segmentation model; The SIFT-based Unet semantic segmentation model is used to perform segmentation processing on the specified oral image data.

[0007] In a second aspect, an embodiment of the present invention provides an oral image segmentation device based on a SIFT algorithm, comprising: An acquisition unit, used for acquiring an oral image dataset containing oral mucosal diseases; A learning unit, used for learning a scale-invariant feature transformation algorithm using the oral image data in the oral image data set, and outputting a pixel-level SIFT descriptor vector about the oral image data by the scale-invariant feature transformation algorithm; A construction unit, used for inputting the pixel-level SIFT descriptor vector into a Unet network, and having the Unet output a corresponding segmentation result, thereby constructing a Unet semantic segmentation model based on SIFT; The segmentation unit is used to perform segmentation processing on the specified oral image data using the SIFT-based Unet semantic segmentation model.

[0008] In a third aspect, an embodiment of the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the oral image segmentation method based on the SIFT algorithm as described in the first aspect is implemented.

[0009] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the oral image segmentation method based on the SIFT algorithm as described in the first aspect is implemented.

[0010] The embodiment of the present invention provides an oral image segmentation method, device, computer equipment and storage medium based on SIFT algorithm, the method comprising: obtaining an oral image data set containing oral mucosal diseases; using the oral image data in the oral image data set to learn the scale-invariant feature transformation algorithm, and the scale-invariant feature transformation algorithm outputs a pixel-level SIFT descriptor vector about the oral image data; inputting the pixel-level SIFT descriptor vector into a Unet network, and the Unet outputs the corresponding segmentation result, thereby constructing a Unet semantic segmentation model based on SIFT; and using the Unet semantic segmentation model based on SIFT to perform segmentation processing on the specified oral image data. The embodiment of the present invention uses the scale-invariant feature transformation algorithm to calculate the local feature descriptor for each pixel in the oral image data, and automatically extracts the features related to the lesion, and then combines the Unet network to realize the pixel-level segmentation of the lesion area of ​​various oral mucosal diseases, so as to improve the segmentation accuracy of the oral image, solve the problem of lesion segmentation of the oral mucosal disease image, thereby significantly improving the diagnostic accuracy and treatment efficiency of oral mucosal diseases, and has good clinical application value and prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0012] Figure 1 A flow chart of an oral image segmentation method based on SIFT algorithm provided in an embodiment of the present invention; Figure 2 A schematic diagram of a sub-process of an oral image segmentation method based on a SIFT algorithm provided in an embodiment of the present invention; Figure 3 A schematic diagram of a pixel-level SIFT descriptor vector in an oral image segmentation method based on a SIFT algorithm provided in an embodiment of the present invention; Figure 4 A schematic diagram of a model architecture in an oral image segmentation method based on a SIFT algorithm provided in an embodiment of the present invention; Figure 5 A schematic diagram of a segmentation test in an oral image segmentation method based on a SIFT algorithm provided in an embodiment of the present invention; Figure 6 A schematic block diagram of an oral image segmentation device based on a SIFT algorithm provided in an embodiment of the present invention; Figure 7 A sub-schematic block diagram of an oral image segmentation device based on the SIFT algorithm provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0013] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0014] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0015] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0016] It should be further understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0017] See below Figure 1 The embodiment of the present invention provides an oral image segmentation method based on SIFT algorithm, which specifically includes: steps S101 to S104.

[0018] Step S101, obtaining an oral image dataset containing oral mucosal diseases; Step S102, using the oral image data in the oral image data set to learn a scale-invariant feature transformation algorithm, and the scale-invariant feature transformation algorithm outputs a pixel-level SIFT descriptor vector for the oral image data; Step S103, inputting the pixel-level SIFT descriptor vector into the Unet network, and having the Unet output the corresponding segmentation result, thereby constructing a Unet semantic segmentation model based on SIFT; Step S104: Use the SIFT-based Unet semantic segmentation model to perform segmentation processing on the specified oral image data.

[0019] In this embodiment, oral image data containing oral mucosal diseases are first collected to obtain a corresponding oral image data set. Then, based on the oral image data set, the scale-invariant feature transform algorithm (i.e., SIFT algorithm) is trained and learned, and the SIFT algorithm calculates a pixel-level SIFT descriptor for each pixel of the oral image data, and then the pixel-level SIFT descriptor is segmented using the Unet network to obtain a segmentation result of the oral image data. Through the above training process, a SIFT-based Unet semantic segmentation model can be constructed, and the SIFT-based Unet semantic segmentation model can be used to perform oral mucosal disease lesion segmentation on the specified oral image data.

[0020] This embodiment uses a scale-invariant feature transformation algorithm to calculate the local feature descriptor for each pixel in the oral image data, and automatically extracts the features related to the lesions, and then combines the Unet network to achieve pixel-level segmentation of the lesion areas of various oral mucosal diseases. This can improve the segmentation accuracy of oral images and solve the problem of lesion segmentation in oral mucosal disease images, thereby significantly improving the diagnostic accuracy and treatment efficiency of oral mucosal diseases, and has good clinical application value and prospects.

[0021] In one embodiment, the step S101 includes: Acquire oral image data containing oral mucosal diseases; Performing data preprocessing on the oral image data; wherein the data preprocessing includes data desensitization processing and data enhancement processing; Labelme software is used to annotate the oral image data, and the oral image dataset is constructed based on the annotated oral image data.

[0022] When acquiring an oral image data set, this embodiment first collects the corresponding oral image data and performs data preprocessing, such as data desensitization, data enhancement, etc., on the data, and then labels the oral image data using Labelme software or other labeling tools to construct the oral image data set. Furthermore, a cross-validation method can be used to divide the oral image data set into a training set and a test set to facilitate subsequent model training and testing.

[0023] In practical applications, oral mucosal case images during the clinical diagnosis and treatment of oral mucosa can be collected, and all collected images can be desensitized and used in .jpg format. Specifically, first determine the data source and select it, including: (1) Sample size calculation, adjustment and allocation: Sample size estimation is a key step in experimental design, but there is currently no standard method for determining sample size estimation for clinical validation studies of deep learning models. Currently, the estimation of the amount of labeled data required to train a classifier to achieve the expected performance is mainly based on prior experience with similar models and problems or unverified heuristic methods. This example uses a model-independent general algorithm for sample size calculation for machine learning clinical validation studies (SSAML), which calculates the minimum sample size to be 800; (2) Inclusion criteria: cases with clinical diagnosis including oral leukoplakia, oral lichen planus, and oral submucous fibrosis; therefore, cases with complete intraoral white light photographs before and after the above diagnosis and treatment and pathology reports were selected; (3) Exclusion criteria: Images with artifacts, abnormal exposure, etc. that affect recognition and labeling were excluded.

[0024] Secondly, the lesion site is annotated. All the collected training data can be annotated manually. For example, doctors use Labelme software (version 5.5.0) to accurately annotate the disease contour through smooth and continuous curves based on the pathology report and clinical diagnosis, and label the marked disease contour with the corresponding diagnosis according to the diagnosis name. Specifically, it includes: (1) Consistency test: First, a senior dentist and a resident doctor can label the collected data separately, compare the labeling results (including diagnosis and disease boundaries), and then conduct a consistency test; the resident doctor re-labels the data two weeks after the first labeling, and compares the two labeling results to achieve intra-group consistency test; (2) Formal labeling: Two experienced dental residents can label all training data, and re-label all labeled data two weeks later to ensure the stability of the labeling by the same resident. Data that pass the consistency test can be included in deep learning. Data that fail the consistency test can be re-labeled by a third senior dental resident. If data still fail the consistency test, the data will be discarded.

[0025] According to the results of physician annotation, among the 838 images in the training set, validation set, and test set, there are 523 images of oral lichen planus (OLP), 201 images of oral leukoplakia (OLK), and 114 images of oral submucous fibrosis (OSF). The actual composition of the dataset is shown in Table 1.

[0026] Table 1

[0027] In addition, in order to improve the generalization ability and robustness of the model in the oral mucosal disease segmentation task, this embodiment designs and implements a series of data preprocessing and enhancement strategies, aiming to simulate a variety of possible image change conditions in the real world, thereby enhancing the model's adaptability to various complex image environments. Specifically, it includes the following aspects: Resizing: Resize the original images to a fixed size to ensure the consistency of the input data and provide standardized training samples for the model; Random affine transformation: simulates the geometric changes of the image through small-angle rotation and shearing (maximum rotation angle is 5°, shearing range is ±5°); Random rotation: randomly rotate the image within the range of ±90° to increase the model's adaptability to different directions; Random flip: flip the image horizontally and vertically with a probability of 50% to enhance the model's robustness to data with inconsistent directions; Color adjustment: Random sharpness adjustment: By setting the sharpness factor to 2.0, the image clarity is randomly enhanced to help the model capture more detail information; Random histogram equalization: enhance the contrast of the image and improve the model's adaptability to low-contrast images; Random automatic contrast adjustment: optimizes the dynamic range of the image and enhances the performance in scenes with changing contrast; Blur processing: Use Gaussian blur (kernel size is (3, 5), standard deviation varies randomly between 0.1 and 2) to simulate the blur effect of the image and improve the model's tolerance to real imaging blur. Random color jitter: Apply color jitter with a probability of 10% to simulate the color changes of images under different lighting conditions or devices, further enhancing the diversity of the data; Format conversion: Convert image data into tensor form and normalize pixel values ​​to the [0, 1] range to adapt to the input format requirements of the model.

[0028] The above enhancement strategy can introduce diversity in a random manner, which can not only keep the semantic information of the original image unchanged, but also significantly expand the effective sample size of the training data. Through this comprehensive preprocessing and enhancement design, the model can be forced to learn more robust and generalized feature representations, thereby providing strong support for the training and application of the model in complex scenarios, effectively alleviating the overfitting problem, and improving the generalization ability and robustness of the model in actual clinical applications.

[0029] In one embodiment, if Figure 2 As shown, the step S102 includes: steps S201~S204.

[0030] Step S201, acquiring each pixel in the oral image data, and setting a neighborhood area for each pixel; Step S202: for each pixel, calculate the gradient magnitude and gradient direction of the corresponding pixel in the corresponding neighborhood area, and construct a gradient direction histogram based on the gradient magnitude and gradient direction; Step S203, dividing the neighborhood area into grid areas, and obtaining gradient direction histograms of different directions in each grid area; Step S204: splicing the gradient direction histograms in each of the grid areas, and using the splicing result as the pixel-level SIFT descriptor vector.

[0031] Scale-Invariant Feature Transform (SIFT) is a classic image feature extraction algorithm that can extract features in an image that are insensitive to local scale, rotation, and illumination changes. It is widely used in target detection, image matching, and 3D reconstruction. The core content of SIFT includes three main steps: key point detection, direction assignment, and descriptor generation. Its basic construction can be divided into the following: (1) Key point detection: SIFT's key point detection is based on the Difference of Gaussian (DoG) pyramid, and its specific process is as follows: Scale space construction: The input image I(x,y) is filtered by the Gaussian kernel Smoothing to generate scale space : ; Among them, x and y are the horizontal and vertical coordinates of the pixel on the plane, respectively. is the standard deviation, indicating the degree of smoothing, and the larger Will result in a more blurred image. The Gaussian kernel Defined as: ; Difference of Gaussian pyramid (DoG) calculation: Difference the Gaussian blurred images of adjacent scales to obtain the Gaussian difference image: ; Where k is the scale factor, It is usually taken as , where s is the number of scales in each octave.

[0032] Extreme value detection: exist In the algorithm, local extreme points are detected by comparing the values ​​of a pixel with the 26 pixels in its spatial and scale neighborhood. These extreme points are used as candidate key points, and are further filtered by suppressing edge responses and low-contrast points to obtain stable key points; (2) Gradient direction assignment: For each detected key point, the gradient magnitude m(x,y) and gradient direction of the local neighborhood are calculated with it as the center : ; ; Among them, x+1 means the current pixel position is shifted one unit to the right, x-1 means the current pixel position is shifted one unit to the left, y+1 means the current pixel position is shifted one unit downward, and y-1 means the current pixel position is shifted one unit upward. Then, a gradient direction histogram is constructed, and the main direction is used as the main direction of the key point to ensure that the feature is rotationally invariant; (3) Descriptor generation: Divide the key point neighborhood into 4×4 grid areas (Cells), and the size of each Cell is , where N is the side length of the key point neighborhood, in pixels. In each cell, the gradient direction histograms in 8 directions are counted, and the direction components are aligned and corrected by the main direction of the key point. Finally, the histograms of all cells are spliced ​​to generate a 128-dimensional descriptor M: M = 4 × 4 × 8 = 128; This descriptor comprehensively represents the local features of the key point neighborhood and shows remarkable robustness in feature matching and modeling.

[0033] In order to model the image features more finely, this embodiment applies the SIFT descriptor to the pixel level. This method generates a SIFT descriptor at each pixel position, making it a feature expression method covering the entire image, namely the pixel-level SIFT descriptor. The specific process is as follows: (1) Pixel neighborhood definition: For each pixel p(x,y) in the image, define a neighborhood region R(x,y) of size N×N, where N is a preset parameter (for example, set N=8): ; Among them, i and j are the horizontal and vertical offsets relative to the center pixel (x, y), respectively. Through different values ​​of i and j, each pixel in the neighborhood can be represented; (2) Gradient information calculation: In the pixel neighborhood, calculate the gradient magnitude m(x,y) and direction of all pixels : ; ; (3) Local partition and orientation histogram: Divide R(x,y) into 4×4 cells, and count the gradient direction histograms of the 8 directional components in each cell: ; Among them, d=1,2,...,8 represents the index of the direction bin, Represents the gradient direction histogram component of the corresponding direction bin in the kth Cell. k is the pixel set of the kth Cell, bin(d) represents the range of the bin in the dth direction of the gradient direction histogram, is the gradient direction ,and Indicates that the gradient direction of pixel (i, j) belongs to the range of the d-th direction bin, and m(i, j) is the gradient magnitude of pixel (i, j).

[0034] Then the gradient direction histograms of all cells are concatenated to form a 128-dimensional feature vector. Each pixel generates an independent 128-dimensional descriptor, which is finally organized into a feature map with 128 channels, each channel corresponding to a directional component of the gradient direction histogram.

[0035] In one embodiment, the step S103 includes: The VGG16 network is used as the backbone network of the Unet network, and the input pixel-level SIFT descriptor vector is segmented; Alternatively, a ResNet50 network is used as the backbone network of the Unet network, and the input pixel-level SIFT descriptor vector is segmented.

[0036] The Backbone network of the UNet network described in this embodiment uses the VGG16 network or the ResNet50 network, and its specific configuration is shown in Table 2.

[0037] Table 2

[0038] Among them, the encoder vector dimension shows the model's ability to extract features at each downsampling stage. Although different Backbone networks (VGG16 or ResNet50) have the same spatial resolution at each layer, the change in the number of channels reflects the complexity of the network in capturing features at different levels. The module depth represents the number of stacked layers of each module in the encoder and decoder. The increase in the number of stacked layers helps to gradually refine the features and increase the amount of information without changing the spatial size of the feature map. The decoder vector dimension determines the richness of the feature map information fusion, especially through upsampling and feature splicing operations, to restore the spatial details of the image and gradually convert it into the final segmentation output.

[0039] The typical image segmentation framework takes image pixels as input and outputs the classification of image pixels. This embodiment uses the UNet model of the convolutional neural network architecture and combines it with the Scale-Invariant Feature Transform (SIFT) algorithm to calculate a local feature descriptor for each image pixel. The descriptor can provide the pixel's position, scale, and rotation-invariant feature vector, such as Figure 3 As shown. The SIFT descriptors of all pixels of the original image are combined to generate a new image with SIFT features, and the new image is input into UNet after feature extraction and dimensionality reduction. Through this process, UNet can directly learn the descriptor vector of the pixel when downsampling, so as to understand the spatial relationship between the pixel and the adjacent vector. Compared with learning and segmenting directly from the image, this embodiment can enhance the understanding of the spatial relationship between pixels, so as to better utilize feature attributes to improve the recognition of boundary attributes. Specifically, the descriptor feature vector encodes the gradient direction frequency of the area around each pixel, enabling UNet to learn more spatial dimensional features of pixels while keeping the resolution of the input image unchanged. Finally, the framework uses the SIFT property to integrate local rotation invariance as part of learning.

[0040] It is also necessary to point out here that the advantage of pixel-level SIFT descriptor vector mapping lies in its global coverage and fine-grained characteristics. When combined with UNet, the pixel-level SIFT descriptor vector is used as the network input, which can expand the receptive field of the UNet network and embed prior information about the gradient direction. This feature improves network performance in the following aspects: (1) It improves the robustness to local rotation and scale changes of the image; (2) It significantly enhances the ability to capture contextual information, which helps to learn the interaction features of global and local information; (3) Even when the amount of data is limited, it can still be effectively generalized, improving the robustness and accuracy of the model.

[0041] Since the SIFT algorithm processes the oral image data at the pixel level, the number of channels will be large, so the image can be reduced in dimension using 1*1 convolution and then input into the UNet network. Figure 4 shown.

[0042] In one embodiment, the oral image segmentation method based on the SIFT algorithm further includes: The Adam optimizer is used to optimize the SIFT-based Unet semantic segmentation model, and the initial learning rate is set to 1. e-4 , the learning rate decay coefficient is 1 e-2 ; The Focal loss function and the Dice loss function are combined to perform feedback update on the SIFT-based Unet semantic segmentation model.

[0043] During the optimization training process, this embodiment adopts some strategies to improve the performance of the model. For example, the optimizer is Adam, and the initial learning rate is set to 1. e-4 , the learning rate decay coefficient is 1 e-2 , which can converge faster in the early stage of training and avoid overfitting in the later stage. The learning rate scheduling adopts the cosine annealing strategy, which can dynamically adjust the learning rate during the training process, effectively balance exploration and utilization, and help the model find the optimal convergence point during the entire training process. In terms of loss function, Focal loss and Dice loss are combined. Focal loss is mainly used to solve the problem of category imbalance, especially when there is a large difference between the background area and the target area, it can enhance the learning of difficult-to-classify areas. Diceloss focuses on optimizing the accuracy of the segmentation boundary to ensure that the generated segmentation results are more detailed and accurate at the edges. Through this combination of loss functions, the model can be comprehensively improved in different optimization directions. Through the above-mentioned optimization strategies and configurations, this embodiment can achieve the best balance between accurate segmentation and training efficiency of the model, thereby further improving the model performance.

[0044] In one embodiment, the oral image segmentation method based on the SIFT algorithm further includes: The performance of the SIFT-based Unet semantic segmentation model is evaluated using semantic segmentation evaluation indicators; wherein the semantic segmentation evaluation indicators include any one or more of the Dice coefficient, average intersection-over-union ratio, average pixel accuracy and precision rate.

[0045] This embodiment uses semantic segmentation evaluation indicators to evaluate the performance of the constructed SIFT-based Unet semantic segmentation model. The semantic segmentation evaluation indicators may specifically include the Dice coefficient, mean intersection over union (mIoU), mean pixel accuracy (mPA) and precision. Among them, the Dice coefficient is used to quantify the similarity between the predicted result and the true annotation. This indicator mainly focuses on the classification of foreground pixels, and the average of the Dice values ​​of each category is taken in the actual calculation. The calculation formula of the Dice coefficient is as follows: ; Among them, Prediction represents the model prediction value, Truth represents the corresponding true value, TP, FP and FN represent the number of true positive, false positive and false negative pixels respectively; Mean intersection over union (mIoU) is the average value of IoU (Intersection over Union) of each category. The calculation formula of IoU is as follows: ; The mean pixel accuracy (mPA) is the average of the pixel accuracy (PA) of each category. PA is an intuitive evaluation indicator in the semantic segmentation task, and its calculation formula is: .

[0046] The above evaluation indicators can reflect the performance of the SIFT-based Unet semantic segmentation model from different angles, and then comprehensively evaluate the segmentation accuracy and effect of the model.

[0047] In actual application scenarios, the SIFT-based Unet semantic segmentation model provided in this embodiment (i.e., PixelSIFT-UNet in Table 3) and the classic semantic segmentation models (U-Net, PSPNet, etc.) were compared in the oral disease segmentation task. In the experiment, two backbone networks (Backbone) of VGG16 and ResNet50 were used for comparison, and the number of parameters and computational complexity (FLOPs) of each model were quantified. The results are shown in Table 3.

[0048] Table 3

[0049] As can be seen from Table 3 above, the PixelSIFT-UNet model has demonstrated excellent performance in this task. The PixelSIFT-UNet using ResNet50 as the backbone network significantly outperforms other models in key performance indicators (mIoU, mPA, and Precision), especially compared to the basic U-Net model, the improvement in mPA is 9.2%. At the same time, the PixelSIFT-UNet model using ResNet50 achieved a Dice coefficient of 0.668, an mIoU of 0.733, an mPA of 0.872, and a Precision of 0.817.

[0050] In order to more intuitively demonstrate the model performance, this embodiment uses the trained PixelSIFT-UNet model to perform segmentation prediction on the oral disease images in the test set. Figure 5 As shown, three images of three categories (1 OLP, 1 OLK, and 1 OSF) are selected in each row, and the original images ( Figure 5 ), the true lesion area (red filling) and the predicted mask (green border) ( Figure 5 The 2nd, 4th, and 6th rows in the figure), where the upper and lower images in the 1st and 2nd rows correspond to each other, the upper and lower images in the 3rd and 4th rows correspond to each other, and the upper and lower images in the 5th and 6th rows correspond to each other. Visual analysis reveals that the PixelSIFT-UNet model can accurately segment the lesion areas of three common oral inflammations (OLP, OLK, and OSF) and provide relatively accurate location and shape information.

[0051] Figure 6 A schematic block diagram of an oral image segmentation device 600 based on a SIFT algorithm provided in an embodiment of the present invention, the device 600 includes: An acquisition unit 601 is used to acquire an oral image dataset containing oral mucosal diseases; A learning unit 602 is used to learn a scale-invariant feature transformation algorithm using the oral image data in the oral image data set, and the scale-invariant feature transformation algorithm outputs a pixel-level SIFT descriptor vector about the oral image data; A construction unit 603 is used to input the pixel-level SIFT descriptor vector into a Unet network, and the Unet outputs a corresponding segmentation result, thereby constructing a SIFT-based Unet semantic segmentation model; The segmentation unit 604 is used to perform segmentation processing on the specified oral image data using the SIFT-based Unet semantic segmentation model.

[0052] In one embodiment, the acquisition unit 601 includes: An image acquisition unit, used for acquiring oral image data including oral mucosal diseases; A data preprocessing unit, used for performing data preprocessing on the oral image data; wherein the data preprocessing includes data desensitization processing and data enhancement processing; The data labeling unit is used to label the oral image data using Labelme software, and construct the oral image data set according to the labeled oral image data.

[0053] In one embodiment, if Figure 7 As shown, the learning unit 602 includes: A neighborhood setting unit 701 is used to obtain each pixel in the oral image data and set a neighborhood area for each pixel; A histogram construction unit 702 is used to calculate the gradient magnitude and gradient direction of the corresponding pixel in the corresponding neighborhood area for each pixel, and construct a gradient direction histogram based on the gradient magnitude and gradient direction; A histogram acquisition unit 703 is used to divide the neighborhood area into grid areas and acquire a gradient direction histogram of different directions in each grid area; The histogram stitching unit 704 is used to stitch the gradient direction histograms in each of the grid areas and use the stitching result as the pixel-level SIFT descriptor vector.

[0054] In one embodiment, the histogram construction unit 702 includes: The gradient calculation unit is used to calculate the gradient amplitude and gradient direction according to the following formula: ; ; Wherein, m(x,y) represents the gradient amplitude, represents the gradient direction, x represents the horizontal coordinate of the pixel on the plane, y represents the vertical coordinate of the pixel on the plane, and I represents the oral image data.

[0055] In one embodiment, the construction unit 603 includes: A first network segmentation unit is used to use a VGG16 network as a backbone network of the Unet network and perform segmentation processing on the input pixel-level SIFT descriptor vector; The second network segmentation unit is used for, or, adopting the ResNet50 network as the backbone network of the Unet network, and performing segmentation processing on the input pixel-level SIFT descriptor vector.

[0056] In one embodiment, the oral image segmentation device 600 based on the SIFT algorithm further includes: The optimization unit is used to optimize the SIFT-based Unet semantic segmentation model using the Adam optimizer and set the initial learning rate to 1. e-4 , the learning rate decay coefficient is 1 e-2 ; An updating unit is used to perform feedback update on the SIFT-based Unet semantic segmentation model in combination with a Focal loss function and a Dice loss function.

[0057] In one embodiment, the oral image segmentation device 600 based on the SIFT algorithm further includes: An evaluation unit is used to perform performance evaluation on the SIFT-based Unet semantic segmentation model using a semantic segmentation evaluation index; wherein the semantic segmentation evaluation index includes any one or more of a Dice coefficient, an average intersection-over-union ratio, an average pixel accuracy rate, and a precision rate.

[0058] Since the embodiments of the apparatus part correspond to the embodiments of the method part, please refer to the description of the embodiments of the method part for the embodiments of the apparatus part, which will not be repeated here.

[0059] The embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed, the steps provided in the above embodiment can be implemented. The storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0060] The embodiment of the present invention also provides a computer device, which may include a memory and a processor, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the steps provided in the above embodiment may be implemented. Of course, the computer device may also include various network interfaces, power supplies and other components.

[0061] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of this application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of this application.

[0062] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises", "comprising" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

Claims

1. An oral image segmentation method based on SIFT algorithm, characterized in that: include: Obtain an oral image dataset containing oral mucosal diseases; Using the oral image data in the oral image data set to learn a scale-invariant feature transformation algorithm, and the scale-invariant feature transformation algorithm outputs a pixel-level SIFT descriptor vector about the oral image data; The pixel-level SIFT descriptor vector is input into the Unet network, and the Unet outputs the corresponding segmentation result, thereby constructing a SIFT-based Unet semantic segmentation model; The SIFT-based Unet semantic segmentation model is used to perform segmentation processing on the specified oral image data.

2. The oral image segmentation method based on SIFT algorithm according to claim 1, characterized in that: The step of obtaining an oral image dataset containing oral mucosal diseases comprises: Acquire oral image data containing oral mucosal diseases; Performing data preprocessing on the oral image data; wherein the data preprocessing includes data desensitization processing and data enhancement processing; Labelme software is used to annotate the oral image data, and the oral image dataset is constructed based on the annotated oral image data.

3. The oral image segmentation method based on SIFT algorithm according to claim 1, characterized in that: The method of learning a scale-invariant feature transformation algorithm by using the oral image data in the oral image data set, and outputting a pixel-level SIFT descriptor vector of the oral image data by the scale-invariant feature transformation algorithm includes: Acquire each pixel in the oral image data, and set a neighborhood area for each pixel; For each pixel, the gradient magnitude and gradient direction of the corresponding pixel are calculated in the corresponding neighborhood area, and a gradient direction histogram is constructed based on the gradient magnitude and gradient direction; Dividing the neighborhood area into grid areas, and obtaining gradient direction histograms of different directions in each of the grid areas; The gradient direction histograms in each of the grid areas are spliced, and the spliced ​​result is used as the pixel-level SIFT descriptor vector.

4. The oral image segmentation method based on SIFT algorithm according to claim 3, characterized in that: For each pixel, the gradient magnitude and gradient direction of the corresponding pixel are calculated in the corresponding neighborhood area, and a gradient direction histogram is constructed based on the gradient magnitude and gradient direction, including: The gradient magnitude and gradient direction are calculated according to the following formula: ； ； in, represents the gradient amplitude, represents the gradient direction, x represents the horizontal coordinate of the pixel on the plane, y represents the vertical coordinate of the pixel on the plane, and I represents the oral image data.

5. The oral image segmentation method based on SIFT algorithm according to claim 1, characterized in that: The pixel-level SIFT descriptor vector is input into the Unet network, and the Unet outputs the corresponding segmentation result, thereby constructing a SIFT-based Unet semantic segmentation model, including: The VGG16 network is used as the backbone network of the Unet network, and the input pixel-level SIFT descriptor vector is segmented; Alternatively, a ResNet50 network is used as the backbone network of the Unet network, and the input pixel-level SIFT descriptor vector is segmented.

6. The oral image segmentation method based on SIFT algorithm according to claim 1, characterized in that: Also includes: The Adam optimizer is used to optimize the SIFT-based Unet semantic segmentation model, and the initial learning rate is set to 1. e-4 , the learning rate decay coefficient is 1 e-2 ; The Focal loss function and the Dice loss function are combined to perform feedback update on the SIFT-based Unet semantic segmentation model.

7. The oral image segmentation method based on SIFT algorithm according to claim 1, characterized in that: Also includes: The performance of the SIFT-based Unet semantic segmentation model is evaluated using semantic segmentation evaluation indicators; wherein the semantic segmentation evaluation indicators include any one or more of the Dice coefficient, average intersection-over-union ratio, average pixel accuracy and precision rate.

8. An oral image segmentation device based on SIFT algorithm, characterized in that: include: An acquisition unit, used for acquiring an oral image dataset containing oral mucosal diseases; A learning unit, used for learning a scale-invariant feature transformation algorithm using the oral image data in the oral image data set, and outputting a pixel-level SIFT descriptor vector about the oral image data by the scale-invariant feature transformation algorithm; A construction unit, used for inputting the pixel-level SIFT descriptor vector into a Unet network, and having the Unet output a corresponding segmentation result, thereby constructing a Unet semantic segmentation model based on SIFT; The segmentation unit is used to perform segmentation processing on the specified oral image data using the SIFT-based Unet semantic segmentation model.

9. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for oral image segmentation based on the SIFT algorithm as claimed in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the oral image segmentation method based on the SIFT algorithm as described in any one of claims 1 to 7 is implemented.

Images