Oral mucosal disease segmentation method and system based on non-salient features

Abstract

The present invention discloses a method and system for segmenting oral mucosal diseases based on non-significant features, including: constructing a segmentation model based on a low-level feature comparison submodule, a logical semantic relationship submodule, an encoder and a decoder; segmenting the preprocessed image to obtain a training set and a test set; inputting the training set into the segmentation model for training, and testing the segmentation model based on the test set to obtain an optimal segmentation model; preprocessing the medical oral image collected again, and inputting the preprocessed image into the optimal segmentation model to obtain a segmented image. The present invention increases the feature differences of different levels between oral lesions and non-lesions through a low-level feature comparison submodule and a logical semantic relationship submodule, thereby improving the segmentation accuracy of non-significant feature areas. And through the feature hierarchy module in the encoder, the input features are evenly divided into several groups along the channel dimension, and oral information is extracted from different perspectives, thereby reducing the number of model parameters.

Description

Technical Field

[0001] The invention belongs to the technical field of image segmentation, and relates to an oral mucosal disease segmentation method and system based on non-significant features. Background Art

[0002] In medical images such as oral cavity, the features of the lesion area are similar to those of the surrounding tissues and organs, that is, there are many non-significant features. The disguised object detection (COD) technology is a technology designed to locate the disguised object from the background. Therefore, the COD technology can well improve the segmentation performance of the non-significant feature area. In the early COD technology, a detection and tracking strategy for discovering and tracking disguised objects was proposed using manually annotated features such as color, intensity, shape, direction, and edge, integrating multiple types of features to represent the disguised object. However, due to the limited expressive power of manual features, these early models often fail to achieve satisfactory performance. To solve this problem, several deep learning-based methods have been developed and have achieved good performance in camouflaged target detection. However, the better the segmentation performance, the more and more complex the model will be, and the deeper the network will be. However, due to limited computing resources, these models are not suitable for actual clinical environments. Therefore, the segmentation model needs to make a good trade-off between performance and model parameters, focusing on designing lightweight and high-performance architectures under resource constraints. FDDWNet uses decomposition-dilated deep separable convolution to learn feature representations from receptive fields of different sizes with fewer model parameters, but ignores the inherent characteristics of semantic segmentation, resulting in poor segmentation accuracy. UTNet is a simple and powerful hybrid Transformer architecture that integrates self-attention into convolutional neural networks for enhanced medical image segmentation, but the learning of non-salient features is not in place and the segmentation accuracy is low. UTNetV2 is an improvement on UTNet. It is a hybrid multi-level structure that uses deep separable convolution as the projection and feedforward network of the transformer block, introduces local modeling, reduces the dependence on big data, but the number of model parameters is still too large. Relevant scholars combined MLP with UNet and proposed UNeXt, which significantly reduced the number of parameters while ensuring segmentation performance, but the visual effect of segmentation is average. At the same time, the lightweight MALUNet achieves competitive skin lesion segmentation performance with low parameter cost and computational complexity, but its segmentation performance in non-salient feature areas is not high. Summary of the invention

[0003] The purpose of the present invention is to solve the problem that the segmentation model in the prior art cannot balance the performance and model parameters, and to provide an oral mucosal disease segmentation method and system based on non-significant features.

[0004] In order to achieve the above object, the present invention adopts the following technical solutions: Oral mucosal disease segmentation methods based on non-significant features include: Preprocessing the collected medical oral images; Construct a segmentation model based on the low-level feature comparison submodule, the logical semantic relationship submodule, the encoder and the decoder; Segment the preprocessed images to obtain training sets and test sets; The training set is input into the segmentation model for training, and the segmentation model is tested based on the test set. When the preset conditions are met, the optimal segmentation model is obtained; The medical oral images collected again are preprocessed, and the preprocessed images are input into the optimal segmentation model to obtain a segmented image.

[0005] A further improvement of the present invention is: Furthermore, the collected medical oral images are preprocessed, specifically, the medical oral images are cropped, resampled, and data normalized.

[0006] Furthermore, the low-level feature contrast submodule extracts low-level features through different receptive fields and cross-space-channel attention mechanisms, and calculates differences to improve the low-level feature contrast, specifically: After inputting the low-level features Fin of the oral image, different receptive fields are divided into local and context branches to extract features of the oral image; After passing through different receptive fields, they are divided according to width and height, downsampled through the average pooling layer, and the sampling results are connected and passed through a 1×1 convolution layer, batch normalization and Swish activation function to obtain the processed feature F mid ; F mid Divide into F h mid ∈C×h×1 and F w mid ∈C×1×w and then pass through a 1×1 convolution layer and a Sigmoid activation function, and multiply the two results with the features before average pooling and output; After local branching, the output feature F local , after the context branch, the output feature F context ; F local and F context After subtraction, instance normalization and Leaky ReLU are performed to obtain the contrast feature F at this level. contrast , and finally output the low-level contrast features in series; Among them, the local receptive field is a 3×3 convolution calculation with an expansion rate of 1; the context receptive field is a 3×3 convolution calculation with an expansion rate of 4.

[0007] Furthermore, the logical semantic relationship submodule includes four branches, each of which contains a series of convolutional layers with different kernel sizes and numbers, representing different receptive fields; the information of all branches is integrated to obtain comprehensive semantic information, specifically: The first branch first passes through a 1×1 convolutional layer and then batch normalization; The second branch first passes through a 1×1 convolution layer, batch normalization and GeLU activation function, then passes through a 7×7 convolution layer, batch normalization and GeLU activation function, and finally passes through a 3×3 convolution layer and batch normalization; The third and fourth branches first pass through a 1×1 convolution layer, batch normalization and GeLU activation function, then pass through two 7×7 convolution layers, batch normalization and GeLU activation function respectively, and finally pass through a 3×3 convolution layer and batch normalization; The outputs of the above four branches are connected, and a 3×3 convolution layer and batch normalization are used. Then they are added with the outputs of a 1×1 convolution layer and batch normalization, and processed through a Leaky ReLU activation function to obtain the final output features.

[0008] Furthermore, the data processed by the logical semantic relationship submodule is passed to the encoder for downsampling in sequence; the encoder has five layers, and each of the first two layers includes two 3×3 convolutional layers and a maximum pooling layer; the first 3×3 convolutional layer is used to extract features; the second 3×3 convolutional layer is used to further extract features; the maximum pooling layer is used for downsampling to reduce the spatial dimension of the features; the last three layers constitute a feature hierarchy module.

[0009] Furthermore, the workflow of the feature hierarchy module is as follows: the input features are group normalized and evenly divided into 4 groups along the channel dimension; The first three groups correspond to the features of the height-width xy axis, channel-height zx axis, and channel-width zy axis respectively; after randomly initializing the learnable tensors pxy, pzx, and pzy, they are respectively subjected to bilinear interpolation and depthwise separable convolution; then the features of different axes and the learnable tensors are divided into three groups for Hadamard Product operation; For the fourth group, only depth-wise separable convolution is used on the feature map, i.e., convolution is performed sequentially through a 1×1 convolution layer, a GeLU activation function, and a 3×3 convolution layer; The outputs of the above four groups are connected along the channel dimension and group normalized; then the feature information is integrated through a 3×3 convolution layer, a GeLU activation function and a 1×1 convolution layer.

[0010] Furthermore, the decoder workflow is as follows: the downsampled data is passed into the decoder for upsampling operations in sequence; the first layer is a feature hierarchy module, and each of the remaining layers is composed of two 3×3 convolutions and one bilinear interpolation.

[0011] Furthermore, the optimal segmentation model is obtained, which is: (1) Set the initial hyperparameters of the segmentation model; (2) Input the preprocessed training set into the segmentation model to start the initial training process of the model; (3) During the training process, monitor the changes in the data set and dynamically adjust the hyperparameters to optimize the learning efficiency of the model and improve the model's adaptability to the data; (4) After each iteration, check the loss function value of the model and record the results of each iteration; (5) When the number of training reaches the preset maximum number of iterations, the model training process will be stopped; (6) After the training is completed, observe the models saved during all iterations, compare their loss function values ​​based on the test set, and find and save the model with the smallest loss function value as the optimal segmentation model.

[0012] Oral mucosal disease segmentation system based on non-significant features, including: A preprocessing module, which preprocesses the collected medical oral images; A construction module, wherein the construction module constructs a segmentation model based on a low-level feature comparison submodule, a logical semantic relationship submodule, an encoder, and a decoder; A segmentation module, wherein the segmentation module segments the preprocessed image to obtain a training set and a test set; A training module, wherein the training module inputs the training set into the segmentation model for training, and tests the segmentation model based on the test set, and when the preset conditions are met, the optimal segmentation model is obtained; The acquisition module preprocesses the medical oral image collected again, and inputs the preprocessed image into an optimal segmentation model to obtain a segmented image.

[0013] Compared with the prior art, the present invention has the following beneficial effects: The present invention constructs a segmentation model through a low-level feature comparison submodule, a logical semantic relationship submodule, an encoder and a decoder, and trains the segmentation model to obtain an optimal segmentation model, thereby obtaining a segmented image. The present invention increases the feature differences of different levels between oral lesions and non-lesions through a low-level feature comparison submodule and a logical semantic relationship submodule, thereby improving the segmentation accuracy of non-significant feature areas. In addition, the feature hierarchy module in the encoder divides the input features into several groups along the channel dimension, extracts and represents oral information from different perspectives, greatly reduces the number of model parameters, and improves the performance of the segmentation model and the balance of model parameters. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0015] Figure 1 Schematic diagram of the process of the oral mucosal disease segmentation method based on non-significant features of the present invention; Figure 2 Schematic diagram of the structure of the oral mucosal disease segmentation system based on non-significant features of the present invention; Figure 3 This is the segmentation model diagram of oral mucosal diseases based on non-significant features; Figure 4 It is a schematic diagram of the structure of the low-level feature comparison submodule; Figure 5 It is a schematic diagram of the structure of the logical semantic relationship submodule; Figure 6 Schematic diagram of the feature hierarchy module structure. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0017] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0019] In the description of the embodiments of the present invention, it should be noted that if the terms "upper", "lower", "horizontal", "inner", etc. indicate an orientation or positional relationship based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship in which the product of the invention is usually placed when in use, it is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0020] In addition, if the term "horizontal" appears, it does not mean that the component must be absolutely horizontal, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", which does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0021] In the description of the embodiments of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal connection of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0022] The present invention is further described in detail below in conjunction with the accompanying drawings: The present invention discloses a method for segmenting oral mucosal diseases based on non-significant features, comprising: S101, preprocessing the collected medical oral images; Perform image cropping, resampling and data normalization on medical oral images.

[0023] S102, constructing a segmentation model based on the low-level feature comparison submodule, the logical semantic relationship submodule, the encoder and the decoder; The low-level feature contrast submodule extracts low-level features through different receptive fields and cross-space-channel attention mechanisms, and calculates differences to improve the low-level feature contrast, specifically: After inputting the low-level features Fin of the oral image, different receptive fields are divided into local and context branches to extract features of the oral image; After passing through different receptive fields, they are divided according to width and height, downsampled through the average pooling layer, and the sampling results are connected and passed through a 1×1 convolution layer, batch normalization and Swish activation function to obtain the processed feature F mid ; F mid Divide into F h mid ∈C×h×1 and F w mid ∈C×1×w and then pass through a 1×1 convolution layer and a Sigmoid activation function, and multiply the two results with the features before average pooling and output; After local branching, the output feature F local , after the context branch, the output feature F context ; F local and F context After subtraction, instance normalization and Leaky ReLU are performed to obtain the contrast feature F at this level. contrast , and finally output the low-level contrast features in series; Among them, the local receptive field is a 3×3 convolution calculation with an expansion rate of 1; the context receptive field is a 3×3 convolution calculation with an expansion rate of 4.

[0024] The logical semantic relationship submodule consists of four branches, each of which contains a series of convolutional layers with different kernel sizes and numbers, representing different receptive fields. The information of all branches is integrated to obtain comprehensive semantic information, specifically: The first branch first passes through a 1×1 convolutional layer and then batch normalization; The second branch first passes through a 1×1 convolution layer, batch normalization and GeLU activation function, then passes through a 7×7 convolution layer, batch normalization and GeLU activation function, and finally passes through a 3×3 convolution layer and batch normalization; The third and fourth branches first pass through a 1×1 convolution layer, batch normalization and GeLU activation function, then pass through two 7×7 convolution layers, batch normalization and GeLU activation function respectively, and finally pass through a 3×3 convolution layer and batch normalization; The outputs of the above four branches are connected, and a 3×3 convolution layer and batch normalization are used. Then they are added with the outputs of a 1×1 convolution layer and batch normalization, and processed through a Leaky ReLU activation function to obtain the final output features.

[0025] The data processed by the logical semantic relationship submodule is passed into the encoder for downsampling operation in sequence; the encoder has five layers, and each of the first two layers includes two 3×3 convolution layers and a maximum pooling layer; the first 3×3 convolution layer is used to extract features; the second 3×3 convolution layer is used to further extract features; the maximum pooling layer is used for downsampling to reduce the spatial dimension of the features; the last three layers constitute a feature hierarchy module.

[0026] The workflow of the feature hierarchy module is as follows: group normalize the input features and divide the input features into 4 groups along the channel dimension; The first three groups correspond to the features of the height-width xy axis, channel-height zx axis, and channel-width zy axis respectively; after randomly initializing the learnable tensors pxy, pzx, and pzy, they are respectively subjected to bilinear interpolation and depthwise separable convolution; then the features of different axes and the learnable tensors are divided into three groups for Hadamard Product operation; For the fourth group, only depth-wise separable convolution is used on the feature map, i.e., convolution is performed sequentially through a 1×1 convolution layer, a GeLU activation function, and a 3×3 convolution layer; The outputs of the above four groups are connected along the channel dimension and group normalized; then the feature information is integrated through a 3×3 convolution layer, a GeLU activation function and a 1×1 convolution layer.

[0027] The decoder workflow is as follows: the downsampled data is passed into the decoder for upsampling operations in sequence; the first layer is a feature hierarchy module, and each of the remaining layers is composed of two 3×3 convolutions and one bilinear interpolation.

[0028] S103, segmenting the preprocessed image to obtain a training set and a test set; S104, inputting the training set into the segmentation model for training, and testing the segmentation model based on the test set, and obtaining the optimal segmentation model when the preset conditions are met; (1) Set the initial hyperparameters of the segmentation model, including the learning rate to 0.01, the batch size to 16, and the number of iterations to 100; (2) Input the preprocessed training set into the segmentation model to start the initial training process of the model; (3) During the training process, closely monitor the changes in the data set and dynamically adjust the hyperparameters based on these changes to optimize the learning efficiency of the model and improve the model's adaptability to the data; (4) After each iteration, check the loss function value of the model and record the results of each iteration; (5) When the number of training reaches the preset maximum number of iterations, the model training process will be stopped; (6) After the training is completed, observe the models saved during all iterations and compare their loss function values ​​based on the test set. Find and save the model with the smallest loss function value as the optimal segmentation model to prepare for the subsequent verification and testing phases.

[0029] S105, preprocessing the medical oral image collected again, and inputting the preprocessed image into an optimal segmentation model to obtain a segmented image.

[0030] See also Figure 2 The present invention discloses an oral mucosal disease segmentation system based on non-significant features, comprising: A preprocessing module, which preprocesses the collected medical oral images; A construction module, wherein the construction module constructs a segmentation model based on a low-level feature comparison submodule, a logical semantic relationship submodule, an encoder, and a decoder; A segmentation module, wherein the segmentation module segments the preprocessed image to obtain a training set and a test set; A training module, wherein the training module inputs the training set into the segmentation model for training, and tests the segmentation model based on the test set, and when the preset conditions are met, the optimal segmentation model is obtained; The acquisition module preprocesses the medical oral image collected again, and inputs the preprocessed image into an optimal segmentation model to obtain a segmented image.

[0031] Example: The present invention discloses a method for segmenting oral mucosal diseases based on non-significant features, comprising: Part 1: Preprocessing of collected medical oral images, specifically: image cropping, resampling and data normalization; Image cropping is to crop all medical oral images to non-zero value areas to reduce the computational burden. The core of this process is to find a minimum bounding box in each image. The specific process is as follows: Generate a non-zero mask area and fill the holes in the non-zero mask area; According to the generated non-zero template, determine the size and position of the bounding box used for cropping; The original image and segmentation mask are cropped according to the bounding box and combined together.

[0032] Resampling is to reprocess the input oral image and adjust it to the appropriate size required by the model. Resampling mainly solves the problem that the actual physical space size represented by each voxel in the medical oral image dataset is inconsistent.

[0033] In order for the convolutional neural network to correctly learn spatial semantic information, different image data need to be resized in voxel space, and all data are resampled to the median. The third-order spline interpolation is used for the original image, and the nearest neighbor interpolation is used for the corresponding segmentation mask. Since two-dimensional RGB medical images are used here, the first element is replaced before resampling to keep the 3D slice spacing unchanged. The specific process is as follows: Determine the target space size for resampling, that is, calculate the median value of each dimension spacing; determine the target size of each image based on the target space size; and resize each image.

[0034] Data normalization is to make the grayscale values ​​of each oral image in the training set have the same distribution. Since the dataset here is RGB medical oral images, a data normalization strategy for non-CT images is selected, which only uses the grayscale information of a single image to calculate the mean and standard deviation, and perform z-scoring (i.e., subtract the mean and divide by the standard deviation).

[0035] Furthermore, if the average size of the images is reduced by 1 / 4 or more during the image cropping stage, we choose to perform data normalization only within the mask of non-zero elements, and all values ​​outside the mask are set to 0.

[0036] The second part is to establish a segmentation model, specifically: Step 1: In a two-dimensional RGB medical oral image, the features (including color, shape, texture, etc.) of the lesion area and the surrounding tissues and organs are relatively similar, that is, there are a large number of non-significant feature areas. Since it is difficult for general segmentation methods to accurately segment the lesions in these non-significant feature areas, the present invention designs a contrast relationship module to sequentially learn the local, contextual, and logical semantic information in the oral image, enhance the non-significant feature differences, and accelerate the feature processing process of the subsequent network. Figure 3 After data preprocessing, the oral image is sent to the contrast relationship module, namely the low-level feature contrast submodule and the logical semantic relationship submodule. The specific implementation details of the low-level feature contrast submodule and the logical semantic relationship submodule are as follows: (1) Low-level feature comparison submodule At present, the common low-level feature extraction methods are to extract different low-level features by setting different convolution kernel sizes and numbers in the neural network, but using a simple single branch to extract low-level features cannot well expand the feature differences. In order to solve the above problems, the present invention designs a low-level feature comparison submodule, which enhances the features of oral lesions by suppressing the features of surrounding tissues and organs through local and contextual branches. The low-level feature comparison submodule helps to expand the segmentation mask to potential similar areas, so that the present invention can better discover the main structure and potential boundaries of oral mucosal lesions. Figure 4 , each low-level feature contrast submodule extracts low-level features through different receptive fields (local, contextual) and cross-space-channel attention mechanisms, and calculates differences to improve low-level feature contrast. The specific workflow is as follows: ① Input low-level features F in After that, feature extraction is performed on local and context branches according to different receptive fields (including color, shape and other information).

[0037] ②The local receptive field represents a 3×3 convolution calculation with a dilation rate of 1, while the contextual receptive field represents a 3×3 convolution calculation with a dilation rate of 4.

[0038] ③ After passing through different receptive fields, they are divided according to width and height, downsampled through the average pooling layer, and then connected to the above results, and passed through a 1×1 convolution layer, batch normalization and Swish activation function to obtain the processed feature F mid .

[0039] ④F mid Divide into F h mid ∈C×h×1 and F w mid ∈C×1×w and then pass through a 1×1 convolution layer and a Sigmoid activation function respectively, and the two parts of the results are multiplied with the features before average pooling and then output.

[0040] ⑤After local branching, output feature F local , after the context branch, the output feature F context . local and F context After subtraction, instance normalization and Leaky ReLU are performed to obtain the contrast feature F at this level. contrast , and finally the low-level contrast features are output in series.

[0041] (2) Logical semantic relationship submodule After learning the low-level features that can be directly extracted from the oral dataset, it is also necessary to consider the impact of deeper high-level features such as semantic relationships on the segmentation performance. For example, when oral leukoplakia and teeth are both white foregrounds, and other tissues in the oral cavity are backgrounds, it is necessary to carefully distinguish what is the real oral lesion in the foreground. To this end, the present invention designs a logical semantic relationship submodule to extract logical semantic relationship information from branches of different receptive fields to accurately identify the real foreground and background of oral lesions. Figure 5 , divided into 4 branches, each branch contains a series of convolutional layers with different kernel sizes and numbers, representing different receptive fields. Then, the information of all branches is integrated to obtain comprehensive semantic information using a wider receptive field. The specific workflow is as follows: ①The first branch first passes through a 1×1 convolutional layer and then batch normalization.

[0042] ②The second branch first passes through a 1×1 convolution layer, batch normalization and GeLU activation function, then passes through a 7×7 convolution layer, batch normalization and GeLU activation function, and finally passes through a 3×3 convolution layer and batch normalization.

[0043] ③The third and fourth branches first pass through a 1×1 convolution layer, batch normalization and GeLU activation function, then pass through two 7×7 convolution layers, batch normalization and GeLU activation function respectively, and finally pass through a 3×3 convolution layer and batch normalization.

[0044] ④ Connect the outputs of the above four branches, use a 3×3 convolution layer and batch normalization, then add them together with the outputs of a 1×1 convolution layer and batch normalization, and finally further process them through the Leaky ReLU activation function to obtain the final output features.

[0045] Step 2: The data processed by the logical semantic relationship submodule is passed to the encoder for downsampling operation in sequence; the encoder has five layers, and each of the first two layers includes two 3×3 convolution layers and a maximum pooling layer; the first 3×3 convolution layer is used to extract features; the second 3×3 convolution layer is used to further extract features; the maximum pooling layer is used for downsampling to reduce the spatial dimension of the features; the last three layers constitute the feature hierarchy module. Figure 6 The feature hierarchy module achieves parameter sharing through grouped depthwise separable convolution, thereby reducing the overall parameter amount and complexity of the oral segmentation model. Compared with other methods for reducing model parameter amount, the feature hierarchy module designed in the present invention significantly reduces the model parameter amount while ensuring the segmentation accuracy of oral mucosal diseases by using multiple views. The specific workflow is as follows: ①First, the input features are group normalized and evenly divided into 4 groups along the channel dimension.

[0046] ② The first three groups correspond to the features of the height-width xy axis, channel-height zx axis, and channel-width zy axis respectively. The randomly initialized learnable tensor p xy 、p zx 、p zy Afterwards, bilinear interpolation and depthwise separable convolution are performed respectively. Then the different axis features and learnable tensors are divided into three groups for Hadamard Product operation.

[0047] ③For the fourth group, only depth-wise separable convolution is used on the feature map, that is, a 1×1 convolution layer, a GeLU activation function, and a 3×3 convolution layer are passed in sequence.

[0048] ④Finally, the outputs of the above four groups are concatenated along the channel dimension and group normalized. Another depth-separable convolution is used to integrate information from different views, that is, a 3×3 convolution layer, a GeLU activation function, and a 1×1 convolution layer are sequentially passed.

[0049] Step 3: The downsampled data is passed to the decoder for upsampling operations. Except for the first layer which is a feature hierarchy module, each of the remaining layers is composed of two 3×3 convolutions and a bilinear interpolation. In addition, each corresponding layer in the encoding and decoding process is connected through a jump connection, and then the segmentation results of oral mucosal diseases are output.

[0050] Part 3: Dividing the training set and test set The present invention divides the preprocessed images, and uses 203 images in the data set as the training set and 35 images as the test set. In addition, since the data preprocessing process has strict requirements on the image format, the unique ID of the data set is first generated as 121, then the original oral image is converted into a lossless RGB image, and finally the segmentation mask is converted into a single-channel image with a pixel value of 0 or 1, and the _0000 symbol is added to the end of each file name.

[0051] According to the confusion matrix, the present invention selects sensitivity (Sen), specificity (Spe), Dice similarity coefficient (Dice), 95% Hausdorff Distance (95HD), Params (M), FLOPs (G), and Memory (M) as evaluation indicators of the experiment, and performs five-fold cross validation.

[0052] The input oral image size was resized to 512×512, the weight decay was 0.001, the initial learning rate was 0.00003, the number of iterations in each epoch was 300, and the total training epoch was 100.

[0053] Part 4: Training and testing an oral mucosal disease segmentation model based on non-significant features The present invention is compared with advanced methods such as UNet, Att-UNet, nnUNet, UNeXt, MALUNet, CRnet, EGE-UNet, TransAttUNet, FCN-8, Mask2Former, and Mask2Former. The results are shown in Table 1-2: Table 1: Comparative experimental results on segmentation performance.

[0054]

[0055] As shown in Table 1, the results obtained using the sensitivity, specificity, 95HD, and Dice evaluation indicators are shown, which are mainly used to analyze the segmentation performance of the present invention. Among them, the sensitivity of UNet is 40.51%, the specificity is 98.93%, the 95HD is 23.16mm, and the Dice is 44.71%; the sensitivity of nnUNet is 70.81%, the specificity is 99.49%, the 95HD is 9.89mm, and the Dice is 73.72%. The improved EGE-UNet based on UNet has a sensitivity of 67.97%, a specificity of 99.01%, a 95HD of 13.96mm, and a Dice of 65.61%. Compared with UNet, the sensitivity, specificity, and Dice of EGE-UNet are increased by 27.46%, 0.08%, and 20.9%, respectively, while the 95HD is reduced by 9.2mm. However, the four indicators of EGE-UNet are insufficient compared with nnUNet, and it does not consider the impact of non-significant features on segmentation performance. The present invention introduces a contrast relationship module that enhances feature differences, thereby achieving better segmentation performance. Compared with UNet, the sensitivity, specificity, and Dice of the present invention are increased by 30.49%, 0.63%, and 32.15%, respectively, and 95HD is reduced by 14.28mm; compared with EGE-UNet, the sensitivity, specificity, and Dice of the present invention are increased by 3.03%, 0.55%, and 11.25%, respectively, and 95HD is reduced by 5.08mm; compared with nnUNet, the sensitivity, specificity, and Dice of the present invention are increased by 0.19%, 0.07%, and 3.14%, respectively, and 95HD is reduced by 1.01mm. The results show that the present invention can better enhance the feature differences of different levels in oral mucosal diseases, and the segmentation performance in non-significant lesion areas is significantly better than other methods.

[0056] Table 2: Comparative experimental results on segmentation efficiency.

[0057]

[0058] Table 2 shows the results obtained using the Params, FLOPs, Memory, and Dice evaluation indicators, which are mainly used to analyze the segmentation efficiency of the present invention. Since the backbone used in the present invention is a large model, the present invention significantly reduces the number of model parameters and complexity while ensuring segmentation performance, but it still cannot be compared with lightweight models. Among them, nnUNet's Params are 126.56M, FLOPs are 466.23G, Memory is 353.69M, and Dice is 73.72%; Mask2Former's Params are 215.23M, FLOPs are 473.85G, Memory is 826.02M, and Dice is 63.04%; they all have a large number of model parameters and complexity. The present invention introduces a feature hierarchy module, which significantly reduces the number of model parameters and complexity while maintaining better segmentation performance. Compared with nnUNet, the Params, FLOPs and Memory of the present invention are reduced by 54.68M, 44.51G and 100.05M respectively, and Dice is increased by 3.14%. Compared with Mask2Former, the Params, FLOPs and Memory of the present invention are reduced by 143.35M, 52.13G and 572.38M respectively, and Dice is increased by 13.82%. This shows that the present invention can significantly reduce the number of parameters and complexity of the oral segmentation model, effectively improve the segmentation efficiency, and better retain the segmentation performance.

[0059] Part V: Example 1: Verify the effectiveness of each module in the oral mucosal disease segmentation model based on non-significant features.

[0060] The segmentation model is mainly composed of a contrast relationship module that enhances non-significant feature differences and a feature hierarchy module that reduces the amount of model parameters. This embodiment verifies the importance of each module by deleting one module at a time. As shown in Table 3, after executing Example 1, the results of the sensitivity, specificity, 95HD, Dice, Params, FLOPs, and Memory evaluation indicators are obtained. When nnUNet is used as the backbone, after adding the contrast relationship module, Dice increases by 1.71% and 95HD decreases by 0.79mm. After adding the feature hierarchy module, Params decreases by 54.69M, FLOPs decreases by 51.27G, and Memory decreases by 100.21M. After adding the contrast relationship module and the feature hierarchy module at the same time, the sensitivity, specificity, and Dice increase by 0.19%, 0.06%, and 3.14%, respectively, and 95HD, Params, FLOPs, and Memory decrease by 1.01mm, 54.68M, 44.51G, and 100.05M, respectively. The results show that the contrast relationship module can significantly improve the segmentation performance of non-significant feature areas in oral mucosal diseases, while the feature hierarchy module can effectively reduce the number of parameters of the oral segmentation model and save resource consumption while ensuring the accuracy of lesion segmentation.

[0061] Table 3: Ablation experiment results on contrastive relationship and feature level modules.

[0062]

[0063] Example 2: Verify the validity of each submodule in the comparison relationship module.

[0064] The contrast relationship module is mainly composed of a low-level feature contrast submodule for enhancing low-level feature differences and a logical semantic relationship submodule for enhancing high-level feature differences. This embodiment verifies the importance of each submodule by deleting one submodule at a time. As shown in Table 4, the results of sensitivity, specificity, 95HD, and Dice evaluation indicators are obtained after executing Example 2. When nnUNet is used as the backbone, after adding the low-level feature contrast submodule, Dice increases by 1.51% and 95HD decreases by 0.37mm. After adding the logical semantic relationship submodule, Dice increases by 1.22% and 95HD decreases by 0.11mm. In addition, after adding the contrast relationship module (low-level feature contrast submodule + logical semantic relationship submodule), Dice increases by 1.71% and 95HD decreases by 0.79mm. The results show that the low-level feature contrast submodule can effectively learn local and contextual information, and the logical semantic relationship submodule can effectively learn high-level feature information, jointly enhancing the feature differences of non-significant lesion areas in oral mucosal diseases and improving segmentation performance.

[0065] Table 4: Ablation experiment results on the two sub-modules in the contrastive relationship module.

[0066]

[0067] Example 3: Verify the effectiveness of the feature hierarchy module.

[0068] The number and position of feature level modules also have different effects on the model parameter quantity and complexity. This embodiment verifies the importance of their number by setting 1 and 3 feature level modules in the encoding part respectively. In addition, this embodiment verifies the importance of their position by setting feature level modules in the encoding and decoding parts respectively. As shown in Table 5, the results of the evaluation indicators of Params, FLOPs, Memory, and Dice are obtained after executing Example 3. When nnUNet is used as the backbone, the encoding part adds 1 feature level module, Params decreases by 18.21M, FLOPs decreases by 1.11G, and Memory decreases by 33.34M. After the encoding part adds 3 feature level modules, Params decreases by 54.63M, FLOPs decreases by 20.09G, and Memory decreases by 100.02M. After the encoding part adds 3 feature level modules and the decoding part adds 1 feature level module, Params decreases by 54.69M, FLOPs decreases by 51.27G, Memory decreases by 100.21M, and Dice increases by 1.69%. The results show that the number of feature hierarchy modules is crucial to reducing the number of parameters in the oral segmentation model, and their different locations can significantly affect the segmentation performance of oral lesions.

[0069] Table 5: Ablation experiment results on different positions and numbers in the feature hierarchy module.

[0070]

[0071] The above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, a person skilled in the art should understand that the specific implementation modes of the present invention can still be modified or replaced by equivalents, and any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be included in the scope of protection of the claims of the present invention.

Claims

1. A method for segmenting oral mucosal diseases based on non-significant features, characterized in that: include: Preprocessing the collected medical oral images; Construct a segmentation model based on the low-level feature comparison submodule, the logical semantic relationship submodule, the encoder and the decoder; Segment the preprocessed images to obtain training sets and test sets; The training set is input into the segmentation model for training, and the segmentation model is tested based on the test set. When the preset conditions are met, the optimal segmentation model is obtained; The medical oral images collected again are preprocessed, and the preprocessed images are input into the optimal segmentation model to obtain a segmented image.

2. The oral mucosal disease segmentation method based on non-significant features according to claim 1, characterized in that: The preprocessing of the collected medical oral images specifically includes: performing image cropping, resampling and data normalization processing on the medical oral images.

3. The oral mucosal disease segmentation method based on non-significant features according to claim 2, characterized in that: The low-level feature contrast submodule extracts low-level features through different receptive fields and cross-space-channel attention mechanisms, and calculates differences to improve low-level feature contrast, specifically: After inputting the low-level features Fin of the oral image, different receptive fields are divided into local and context branches to extract features of the oral image; After passing through different receptive fields, they are divided according to width and height, downsampled through the average pooling layer, and the sampling results are connected and passed through a 1×1 convolution layer, batch normalization and Swish activation function to obtain the processed feature F mid ; F mid Divide into F h mid ∈C×h×1 and F w mid ∈C×1×w and then pass through a 1×1 convolution layer and a Sigmoid activation function, and multiply the two results with the features before average pooling and output; After local branching, the output feature F local , after the context branch, the output feature F context ; F local and F context After subtraction, instance normalization and Leaky ReLU are performed to obtain the contrast feature F at this level. contrast , and finally output the low-level contrast features in series; Among them, the local receptive field is a 3×3 convolution calculation with an expansion rate of 1; the context receptive field is a 3×3 convolution calculation with an expansion rate of 4.

4. The oral mucosal disease segmentation method based on non-significant features according to claim 3, characterized in that: The logical semantic relationship submodule includes 4 branches, each of which contains a series of convolutional layers with different kernel sizes and numbers, representing different receptive fields; the information of all branches is integrated to obtain comprehensive semantic information, specifically: The first branch first passes through a 1×1 convolutional layer and then batch normalization; The second branch first passes through a 1×1 convolution layer, batch normalization and GeLU activation function, then passes through a 7×7 convolution layer, batch normalization and GeLU activation function, and finally passes through a 3×3 convolution layer and batch normalization; The third and fourth branches first pass through a 1×1 convolution layer, batch normalization and GeLU activation function, then pass through two 7×7 convolution layers, batch normalization and GeLU activation function respectively, and finally pass through a 3×3 convolution layer and batch normalization; The outputs of the above four branches are connected, and a 3×3 convolution layer and batch normalization are used. Then they are added with the outputs of a 1×1 convolution layer and batch normalization, and processed through a Leaky ReLU activation function to obtain the final output features.

5. The oral mucosal disease segmentation method based on non-significant features according to claim 4, characterized in that: The data processed by the logical semantic relationship submodule is passed into the encoder for downsampling operation in sequence; the encoder has five layers, and each of the first two layers includes two 3×3 convolution layers and a maximum pooling layer; the first 3×3 convolution layer is used to extract features; the second 3×3 convolution layer is used to further extract features; the maximum pooling layer is used for downsampling to reduce the spatial dimension of the features; the last three layers constitute a feature hierarchy module.

6. The oral mucosal disease segmentation method based on non-significant features according to claim 5, characterized in that: The workflow of the feature hierarchy module is as follows: group normalization of the input features and average division of the input features into 4 groups along the channel dimension; The first three groups correspond to the features of the height-width xy axis, channel-height zx axis, and channel-width zy axis respectively; after randomly initializing the learnable tensors pxy, pzx, and pzy, they are respectively subjected to bilinear interpolation and depthwise separable convolution; then the features of different axes and the learnable tensors are divided into three groups for Hadamard Product operation; For the fourth group, only depth-wise separable convolution is used on the feature map, i.e., convolution is performed sequentially through a 1×1 convolution layer, a GeLU activation function, and a 3×3 convolution layer; The outputs of the above four groups are connected along the channel dimension and group normalized; then the feature information is integrated through a 3×3 convolution layer, a GeLU activation function and a 1×1 convolution layer.

7. The oral mucosal disease segmentation method based on non-significant features according to claim 6, characterized in that: The decoder workflow is as follows: the downsampled data is passed into the decoder for upsampling operations in sequence; the first layer is a feature hierarchy module, and each of the remaining layers is composed of two 3×3 convolutions and one bilinear interpolation.

8. The oral mucosal disease segmentation method based on non-significant features according to claim 7, characterized in that: The optimal segmentation model is specifically: (1) Set the initial hyperparameters of the segmentation model; (2) Input the preprocessed training set into the segmentation model to start the initial training process of the model; (3) During the training process, monitor the changes in the data set and dynamically adjust the hyperparameters to optimize the learning efficiency of the model and improve the model's adaptability to the data; (4) After each iteration, check the loss function value of the model and record the results of each iteration; (5) When the number of training reaches the preset maximum number of iterations, the model training process will be stopped; (6) After the training is completed, observe the models saved during all iterations, compare their loss function values ​​based on the test set, and find and save the model with the smallest loss function value as the optimal segmentation model.

9. Oral mucosal disease segmentation system based on non-significant features, characterized in that: include: A preprocessing module, which preprocesses the collected medical oral images; A construction module, wherein the construction module constructs a segmentation model based on a low-level feature comparison submodule, a logical semantic relationship submodule, an encoder, and a decoder; A segmentation module, wherein the segmentation module segments the preprocessed image to obtain a training set and a test set; A training module, wherein the training module inputs the training set into the segmentation model for training, and tests the segmentation model based on the test set, and when the preset conditions are met, the optimal segmentation model is obtained; The acquisition module preprocesses the medical oral image collected again, and inputs the preprocessed image into an optimal segmentation model to obtain a segmented image.

Images