An image recognition system for oral mucosal lesions

Abstract

The present invention relates to image processing technology and provides an image recognition system for oral mucosal lesions, aiming to improve the accuracy and efficiency of oral image processing. The system includes a data acquisition module, a data preprocessing module, an image segmentation module, a lesion area classification module, and a report display and visualization module. The image segmentation module achieves high-precision lesion area segmentation through image pyramid, Sobel edge detection, Otsu threshold segmentation, and local contrast weighted optimization technology. The lesion area classification module combines feature combination, feature interaction, HEMish activation function, and variable dilated causal convolution to accurately classify oral lesion areas. The present invention significantly improves the recognition accuracy, robustness, and adaptability of lesion areas through multi-level and multi-dimensional technical means, solves the problems of low accuracy and unstable recognition in traditional systems, and promotes technological progress in the field of oral mucosal lesion image recognition.

Description

Technical Field

[0001] The present invention relates to image processing technology, in particular to an image recognition system for oral mucosal lesions. Background Art

[0002] With the continuous advancement of science and technology, image recognition and deep learning technologies are increasingly used in the field of medical image analysis; however, traditional oral mucosal image recognition systems still face many challenges; first, existing systems still find it difficult to meet the requirements of high accuracy and high efficiency in the precise segmentation and classification of lesion areas, especially under complex backgrounds and with variable lesion manifestations; second, traditional systems have low sensitivity to local features, especially in details such as edges and textures, which are often ignored, thus affecting the accurate identification of lesion areas; finally, traditional methods rely on relatively simple image segmentation and classification techniques, ignoring the capture of subtle differences between lesion areas, resulting in inaccurate segmentation and classification results; therefore, there is an urgent need for a more efficient and accurate oral mucosal lesion image recognition system to provide more accurate lesion area segmentation and classification results, and to have greater robustness and adaptability. Summary of the Invention

[0003] The present invention provides an image recognition system for oral mucosal lesions, which adopts novel image segmentation and lesion area classification technologies to improve the accuracy and robustness of lesion area recognition. Specifically, the system combines image pyramid, Sobel edge detection, Otsu threshold segmentation and local contrast weighted optimization technology in the image segmentation module, accurately segments the lesion area in the oral image through multi-scale processing and edge detection, and ensures the clarity and accuracy of the segmentation results through local contrast weighted optimization. The system's lesion area classification module further improves the feature extraction and processing capabilities through feature combination, feature interaction, HEMish activation function and variable dilated causal convolution technology, thereby achieving accurate classification of lesion areas. Overall, the present invention ensures efficient and accurate recognition of oral mucosal lesion images through multi-level and multi-dimensional technical means, and has strong robustness and adaptability.

[0004] The present invention provides an image recognition system for oral mucosal lesions, which includes a data acquisition module, a data preprocessing module, an image segmentation module, a lesion area classification module, and a report display and visualization module;

[0005] The data acquisition module uses an endoscope, a high-definition camera, and a digital oral camera to collect images of the oral cavity as raw oral image data; the raw oral image data includes mucosal surface images and lesion area images;

[0006] The data preprocessing module performs image denoising, image enhancement, image normalization, image cropping and image scaling on the original oral image data to generate preprocessed oral image data;

[0007] The image segmentation module combines image pyramid, Sobel edge detection, Otsu threshold segmentation and local contrast weighted optimization technology to build a multi-scale weighted Otsu edge segmentation model. The multi-scale weighted Otsu edge segmentation model is used to process the pre-processed oral image data and generate a multi-scale edge segmentation map.

[0008] The lesion area classification module establishes a convolutional neural network model. By introducing feature combination, feature interaction, HEMish activation function and variable dilated causal convolution to improve the convolutional neural network model, a multi-scale warped-variable convolutional neural network model is constructed. The multi-scale warped-variable convolutional neural network model processes the multi-scale edge segmentation map to generate lesion area classification information.

[0009] The report display and visualization module generates an analysis report on oral lesions based on lesion area classification information and multi-scale edge segmentation maps. The oral lesion analysis report includes classification results, detailed information on the lesion area, and image annotations. The oral lesion analysis report is displayed in a visual interface to view the lesion area, the probability distribution of the classification results, and the annotations on the image, assisting doctors in further observation, data archiving, and historical record comparison.

[0010] The multi-scale weighted Otsu edge segmentation model includes a multi-scale image pyramid unit, a Sobel operator edge detection unit, a multi-scale Otsu threshold unit and a generation unit;

[0011] The multi-scale warped-variable convolutional neural network model includes a fully connected layer 1, a dropout layer, a fully connected layer 2, a residual connection, and an output layer.

[0012] Furthermore, the multi-scale image pyramid unit constructs an image pyramid and decomposes the preprocessed oral image data into five levels of different resolutions, each level containing two scales, to obtain multi-scale image data.

[0013] Furthermore, the Sobel operator edge detection unit applies the Sobel operator to perform edge detection on the multi-scale image data, calculates the horizontal gradient and the vertical gradient, calculates the gradient amplitude according to the horizontal gradient and the vertical gradient, and highlights the edge area through the gradient amplitude.

[0014] Furthermore, the multi-scale Otsu threshold unit calculates the grayscale histogram and inter-class variance of the multi-scale image data, and then introduces local contrast weighted optimization to determine the optimal threshold. According to the optimal threshold, the gradient amplitude and the multi-scale image data are binarized to achieve the segmentation of the foreground and background in the multi-scale image data, and obtain a multi-scale binary image. The formula used is as follows:

[0015] ;

[0016] in, Represents the index of the local area, Indicates the The contrast of a local area, Represents a local area The pixel value of Represents a local area The maximum value of the pixel value, Represents a local area The minimum pixel value, Represents a local area The average value of all pixel values ​​in ;

[0017] ;

[0018] in, represents the candidate threshold, represents the between-class variance, Represent the weights of foreground and background in multi-scale image data, and Represent the mean of foreground and background in multi-scale image data respectively.

[0019] Furthermore, the generation unit refines the edges of the multi-scale binary image, removes noise points, and fills gaps to generate a multi-scale edge segmentation map.

[0020] Furthermore, the lesion area classification module generates lesion area classification information, specifically including the following steps:

[0021] Step D1: Feature combination: perform product combination and pairwise product summation on the multi-scale edge segmentation map to generate combined feature data;

[0022] Step D2: Feature Interaction: The combined feature data is nonlinearly transformed through the ReLU activation function to capture the high-order interaction relationship between the combined feature data and generate rich high-order feature representations; thus enhancing the model's ability to express complex relationships between features.

[0023] Step D3: Feature transformation: Combining the smoothness of Mish, Gaussian function, hyperbolic tangent function and adjustable hyperparameters, we construct the HEMish activation function. We perform linear transformation on the rich high-order feature representations, and then perform nonlinear transformation on them through the HEMish activation function to generate deep feature representations. This further improves the nonlinear expression capability of the features. The formula used is as follows:

[0024] ;

[0025] in, represents linear transformation data, represents the HEMish activation function, represents the hyperbolic tangent function; Represents an adjustable parameter that controls the influence strength of the Gaussian function; Represents an adjustable parameter that controls the rate of decay; represents the Gaussian decay function; Express Perform smooth nonlinear transformations; represents the amplitude factor of the sine function, Represents the frequency factor of the sine function; represents a periodic function, Represents an exponential function, giving the ability to express periodic changes in a nonlinear manner;

[0026] Step D4: Feature Mapping 1: Process the deep feature representation through the fully connected layer 1 to generate the fully connected layer 1 feature data; learn the deep feature representation through the fully connected layer 1 and output the expansion factor control signal;

[0027] Step D5: Causal convolution: Variable dilated causal convolution is used to further capture the long-term dependencies between the feature data of the fully connected layer 1 and generate variable causal feature data.

[0028] Step D6: Batch normalization: Batch normalize the variable causal feature data to reduce the internal covariance changes during training and generate batch normalized feature data;

[0029] Step D7: Dropout processing: The batch normalized feature data is passed through the Dropout layer to randomly discard some neurons to generate Dropout feature data;

[0030] Step D8: Feature Mapping 2: The Dropout feature data is further refined through the fully connected layer 2 to generate the fully connected layer 2 feature data.

[0031] Step D9: Generate classification results: Input the feature data of the fully connected layer 2 into the output layer to generate classification information of the lesion area;

[0032] Step D10: Residual connection: Add a residual connection between the fully connected layer 1 and the fully connected layer 2 to solve the gradient disappearance problem.

[0033] Furthermore, step D5 specifically includes the following steps:

[0034] Step D51: Introduce a variable dilation factor to improve the dilated causal convolution, construct a variable dilated causal convolution, initialize the variable dilation factor, and set the range of the variable dilation factor;

[0035] Step D52: According to the variable dilation factor range and the dilation factor control signal, use dilated causal convolution to change the receptive field of the convolution kernel, capture the long-term dependency between the feature data of the fully connected layer 1, and generate variable causal feature data.

[0036] The beneficial effects achieved by the present invention using the above scheme are as follows:

[0037] The present invention effectively achieves accurate segmentation of oral mucosal lesion images by adopting multi-scale weighted Otsu edge segmentation and local contrast weighted optimization technology. This image segmentation technology can overcome the segmentation inaccuracies of traditional systems under complex backgrounds and variable lesion types. Through multi-scale image pyramid and Sobel edge detection technology, it ensures clear presentation of lesion areas at different resolutions and improves segmentation accuracy. The introduction of this technical means significantly solves the problem of insufficient performance of oral mucosal image recognition systems when facing high-noise or low-contrast images, thereby ensuring that the data after image preprocessing is more suitable for subsequent lesion classification and improving the recognition accuracy of the entire system.

[0038] In addition, the system's lesion area classification module improves the accuracy of lesion area classification by introducing feature combination, feature interaction, and HEMish activation function technology. This module enhances the system's ability to capture complex features and can effectively address the identification of subtle differences in lesion areas. Through these technological improvements, the system can better handle various lesion types, especially the capture of details and the sensitivity to local features have been significantly improved. Compared with traditional systems, the present invention greatly enhances the system's adaptability to different oral images, enabling it to accurately classify and identify various oral mucosal lesions, further improving the reliability and practicality of the oral mucosal image recognition system.

[0039] In summary, the present invention significantly improves the overall performance of the oral mucosal lesion image recognition system by combining advanced image segmentation and classification technologies; the innovative improvement of the multi-scale weighted Otsu edge segmentation technology and the lesion area classification module not only effectively improves the segmentation accuracy of the lesion area, but also enhances the reliability of the classification results, providing a more accurate and efficient technical means for the automatic analysis of oral medical images. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a module schematic diagram of an image recognition system for oral mucosal lesions proposed by the present invention;

[0041] Figure 2 This is a model architecture diagram of the multi-scale warped-variable convolutional neural network model proposed in Example 7. DETAILED DESCRIPTION

[0042] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0043] Example 1, according to Figure 1 , the present invention provides an image recognition system for oral mucosal lesions, the system comprising a data acquisition module, a data preprocessing module, an image segmentation module, a lesion area classification module and a report display and visualization module;

[0044] The data acquisition module uses an endoscope, a high-definition camera, and a digital oral camera to collect images of the oral cavity as raw oral image data; the raw oral image data includes mucosal surface images and lesion area images;

[0045] The data preprocessing module performs image denoising, image enhancement, image normalization, image cropping and image scaling on the original oral image data to generate preprocessed oral image data;

[0046] The image segmentation module combines image pyramid, Sobel edge detection, Otsu threshold segmentation and local contrast weighted optimization technology to build a multi-scale weighted Otsu edge segmentation model. The multi-scale weighted Otsu edge segmentation model is used to process the pre-processed oral image data and generate a multi-scale edge segmentation map.

[0047] The lesion area classification module establishes a convolutional neural network model. By introducing feature combination, feature interaction, HEMish activation function and variable dilated causal convolution to improve the convolutional neural network model, a multi-scale warped-variable convolutional neural network model is constructed. The multi-scale warped-variable convolutional neural network model processes the multi-scale edge segmentation map to generate lesion area classification information.

[0048] The report display and visualization module generates an analysis report on oral lesions based on lesion area classification information and multi-scale edge segmentation maps. The oral lesion analysis report includes classification results, detailed information on the lesion area, and image annotations. The oral lesion analysis report is displayed in a visual interface to view the lesion area, the probability distribution of the classification results, and the annotations on the image, assisting doctors in further observation, data archiving, and historical record comparison.

[0049] The multi-scale weighted Otsu edge segmentation model includes a multi-scale image pyramid unit, a Sobel operator edge detection unit, a multi-scale Otsu threshold unit and a generation unit;

[0050] The multi-scale warped-variable convolutional neural network model includes a fully connected layer 1, a dropout layer, a fully connected layer 2, a residual connection, and an output layer.

[0051] Example 2: This example is based on Example 1. In this example, a multi-scale image pyramid unit constructs an image pyramid, decomposes the preprocessed oral image data into five levels of different resolutions, each level contains two scales, and obtains multi-scale image data.

[0052] Example 3. This example is based on Example 2. In this example, the Sobel operator edge detection unit applies the Sobel operator to perform edge detection on multi-scale image data, calculates the horizontal gradient and the vertical gradient, calculates the gradient amplitude based on the horizontal gradient and the vertical gradient, and highlights the edge area through the gradient amplitude.

[0053] Example 4: This example is based on Example 3. In this example, a multi-scale Otsu threshold unit calculates the grayscale histogram and inter-class variance of the multi-scale image data, and then introduces local contrast weighted optimization to determine the optimal threshold. Based on the optimal threshold, the gradient amplitude and the multi-scale image data are binarized to achieve foreground and background segmentation in the multi-scale image data, thereby obtaining a multi-scale binary image. The formula used is as follows:

[0054] ;

[0055] in, Represents the index of the local area, Indicates the The contrast of a local area, Represents a local area The pixel value of Represents a local area The maximum value of the pixel value, Represents a local area The minimum pixel value, Represents a local area The average value of all pixel values ​​in ;

[0056] ;

[0057] in, represents the candidate threshold, represents the between-class variance, Represent the weights of foreground and background in multi-scale image data, and Represent the mean of foreground and background in multi-scale image data respectively.

[0058] Example 5. This example is based on Example 3. In this example, a multi-scale Otsu threshold unit calculates the grayscale histogram and inter-class variance of the multi-scale image data, determines the optimal threshold, and binarizes the gradient amplitude and the multi-scale image data according to the optimal threshold to achieve segmentation of the foreground and background in the multi-scale image data and obtain a multi-scale binary image.

[0059] Embodiment 6: This embodiment is based on embodiment 4. In this embodiment, the generating unit performs edge thinning, noise removal, and gap filling on the multi-scale binary image to generate a multi-scale edge segmentation map.

[0060] Embodiment 7, according to Figure 2 This embodiment is based on the sixth embodiment. In this embodiment, the process of generating lesion area classification information by the lesion area classification module specifically includes the following steps:

[0061] Step D1: Feature combination: perform product combination and pairwise product summation on the multi-scale edge segmentation map to generate combined feature data;

[0062] Step D2: Feature Interaction: The combined feature data is nonlinearly transformed through the ReLU activation function to capture the high-order interaction relationship between the combined feature data and generate rich high-order feature representations; thus enhancing the model's ability to express complex relationships between features.

[0063] Step D3: Feature transformation: Combining the smoothness of Mish, Gaussian function, hyperbolic tangent function and adjustable hyperparameters, we construct the HEMish activation function. We perform linear transformation on the rich high-order feature representations, and then perform nonlinear transformation on them through the HEMish activation function to generate deep feature representations. This further improves the nonlinear expression capability of the features. The formula used is as follows:

[0064] ;

[0065] in, represents linear transformation data, represents the HEMish activation function, represents the hyperbolic tangent function; Represents an adjustable parameter that controls the influence strength of the Gaussian function; Represents an adjustable parameter that controls the rate of decay; represents the Gaussian decay function; Express Perform smooth nonlinear transformations; represents the amplitude factor of the sine function, Represents the frequency factor of the sine function; represents a periodic function, Represents an exponential function, giving the ability to express periodic changes in a nonlinear manner;

[0066] Step D4: Feature Mapping 1: Process the deep feature representation through the fully connected layer 1 to generate the fully connected layer 1 feature data; learn the deep feature representation through the fully connected layer 1 and output the expansion factor control signal;

[0067] Step D5: Causal convolution: Variable dilated causal convolution is used to further capture the long-term dependencies between the feature data of the fully connected layer 1 and generate variable causal feature data.

[0068] Step D6: Batch normalization: Batch normalize the variable causal feature data to reduce the internal covariance changes during training and generate batch normalized feature data;

[0069] Step D7: Dropout processing: The batch normalized feature data is passed through the Dropout layer to randomly discard some neurons to generate Dropout feature data;

[0070] Step D8: Feature Mapping 2: The Dropout feature data is further refined through the fully connected layer 2 to generate the fully connected layer 2 feature data.

[0071] Step D9: Generate classification results: Input the feature data of the fully connected layer 2 into the output layer to generate classification information of the lesion area;

[0072] Step D10: Residual connection: Add a residual connection between the fully connected layer 1 and the fully connected layer 2 to solve the gradient disappearance problem.

[0073] Embodiment 8: This embodiment is based on embodiment 6. In this embodiment, the process of generating lesion area classification information by the lesion area classification module specifically includes the following steps:

[0074] Step D1: Feature combination: perform product combination and pairwise product summation on the multi-scale edge segmentation map to generate combined feature data;

[0075] Step D2: Feature Interaction: The combined feature data is nonlinearly transformed through the ReLU activation function to capture the high-order interaction relationship between the combined feature data and generate rich high-order feature representations; thus enhancing the model's ability to express complex relationships between features.

[0076] Step D3: Feature transformation: The rich high-order feature representation is linearly transformed, and then nonlinearly transformed through the ReLU activation function to generate a deep feature representation; further improving the nonlinear expression ability of the feature;

[0077] Step D4: Feature Mapping 1: Process the deep feature representation through the fully connected layer 1 to generate the fully connected layer 1 feature data; learn the deep feature representation through the fully connected layer 1 and output the expansion factor control signal;

[0078] Step D5: Causal convolution: The long-term dependencies between the feature data of the fully connected layer 1 are further captured by dilated causal convolution to generate variable causal feature data;

[0079] Step D6: Batch normalization: Batch normalize the variable causal feature data to reduce the internal covariance changes during training and generate batch normalized feature data;

[0080] Step D7: Dropout processing: The batch normalized feature data is passed through the Dropout layer to randomly discard some neurons to generate Dropout feature data;

[0081] Step D8: Feature Mapping 2: The Dropout feature data is further refined through the fully connected layer 2 to generate the fully connected layer 2 feature data.

[0082] Step D9: Generate classification results: Input the feature data of the fully connected layer 2 into the output layer to generate classification information of the lesion area;

[0083] Step D10: Residual connection: Add a residual connection between the fully connected layer 1 and the fully connected layer 2 to solve the gradient disappearance problem.

[0084] Embodiment 9: This embodiment is based on embodiment 7. In this embodiment, step D5 specifically includes the following steps:

[0085] Step D51: Introduce a variable dilation factor to improve the dilated causal convolution, construct a variable dilated causal convolution, initialize the variable dilation factor, and set the range of the variable dilation factor;

[0086] Step D52: According to the variable dilation factor range and the dilation factor control signal, use dilated causal convolution to change the receptive field of the convolution kernel, capture the long-term dependency between the feature data of the fully connected layer 1, and generate variable causal feature data.

[0087] The present invention and its embodiments are described above. Such description is not restrictive. What is shown in the accompanying drawings is only one of the embodiments of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in this field are inspired by it and do not depart from the purpose of the invention, they can creatively design structural methods and embodiments similar to the technical solution, which should all fall within the scope of protection of the present invention.

Claims

1. An image recognition system for oral mucosal lesions, comprising a data acquisition module and a data preprocessing module; the data acquisition module acquires raw oral image data; the data preprocessing module preprocesses the raw oral image data to generate preprocessed oral image data; characterized in that: The system also includes an image segmentation module and a lesion area classification module; The image segmentation module combines image pyramid, Sobel edge detection, Otsu threshold segmentation and local contrast weighted optimization technology to build a multi-scale weighted Otsu edge segmentation model, and processes the pre-processed oral image data through the multi-scale weighted Otsu edge segmentation model to generate a multi-scale edge segmentation map; The lesion area classification module establishes a convolutional neural network model, improves the convolutional neural network model by introducing feature combination, feature interaction, HEMish activation function and variable dilated causal convolution, and constructs a multi-scale warped-variable convolutional neural network model; processes the multi-scale edge segmentation map through the multi-scale warped-variable convolutional neural network model to generate lesion area classification information; The multi-scale weighted Otsu edge segmentation model includes a multi-scale image pyramid unit, a Sobel operator edge detection unit, a multi-scale Otsu threshold unit and a generation unit; The multi-scale warped-variable convolutional neural network model includes a fully connected layer 1, a dropout layer, a fully connected layer 2, a residual connection, and an output layer; The process of generating lesion area classification information by the lesion area classification module specifically includes the following steps: Step D1: Feature combination: perform product combination and pairwise product summation on the multi-scale edge segmentation map to generate combined feature data; Step D2: Feature interaction: The combined feature data is nonlinearly transformed through the ReLU activation function to capture the high-order interaction relationship of the combined feature data and generate rich high-order feature representations; Step D3: Feature transformation: Combining the smoothness of Mish, Gaussian function, hyperbolic tangent function and adjustable hyperparameters, the HEMish activation function is constructed; the rich high-order feature representation is linearly transformed, and then nonlinearly transformed through the HEMish activation function to generate a deep feature representation; Step D4: Feature Mapping 1: Process the deep feature representation through the fully connected layer 1 to generate a fully connected layer 1 feature dataset; learn the deep feature representation through the fully connected layer 1 and output the dilation factor control signal; Step D5: Causal convolution: Capture the long-term dependencies in the fully connected layer 1 feature dataset through variable dilated causal convolution to generate variable causal feature data; Step D6: Batch normalization: Batch normalize the variable causal feature data to reduce the internal covariance changes during training and generate batch normalized feature data; Step D7: Dropout processing: The batch normalized feature data is passed through the Dropout layer to randomly discard some neurons to generate Dropout feature data; Step D8: Feature Mapping 2: Refine the feature representation of the Dropout feature data through the fully connected layer 2 to generate the fully connected layer 2 feature data; Step D9: Generate classification results: Input the feature data of the fully connected layer 2 into the output layer to generate classification information of the lesion area; Step D10: Residual connection: Add a residual connection between fully connected layer 1 and fully connected layer 2.

2. The image recognition system for oral mucosal lesions according to claim 1, characterized in that: The multi-scale image pyramid unit constructs an image pyramid, decomposes the pre-processed oral image data into five levels with different resolutions, each level contains two scales, and obtains multi-scale image data.

3. The image recognition system for oral mucosal lesions according to claim 2, characterized in that: The Sobel operator edge detection unit applies the Sobel operator to perform edge detection on the multi-scale image data, calculates the horizontal gradient and the vertical gradient, calculates the gradient amplitude according to the horizontal gradient and the vertical gradient, and highlights the edge area through the gradient amplitude.

4. The image recognition system for oral mucosal lesions according to claim 3, characterized in that: The multi-scale Otsu threshold unit calculates the grayscale histogram and inter-class variance of the multi-scale image data, then introduces local contrast weighted optimization to determine the optimal threshold. Based on the optimal threshold, the gradient amplitude and the multi-scale image data are binarized to achieve foreground and background segmentation in the multi-scale image data, thereby obtaining a multi-scale binary image.

5. The image recognition system for oral mucosal lesions according to claim 4, characterized in that: The generating unit refines the edge of the multi-scale binary image, removes noise points and fills gaps to generate a multi-scale edge segmentation map.

6. The image recognition system for oral mucosal lesions according to claim 1, characterized in that: The step D5 specifically includes the following steps: Step D51: Introduce a variable dilation factor to improve the dilated causal convolution, construct a variable dilated causal convolution, initialize the variable dilation factor, and set the range of the variable dilation factor; Step D52: According to the variable dilation factor range and the dilation factor control signal, use dilated causal convolution to change the receptive field of the convolution kernel, capture the long-term dependency between the feature data of the fully connected layer 1, and generate variable causal feature data.

Images