Early esophageal squamous carcinoma detection method and system based on deep learning

Abstract

The invention provides an early-stage esophageal squamous carcinoma detection method and system based on deep learning, and relates to the technical field of image processing, and the method comprises the steps: obtaining an esophageal endoscope image, carrying out the enhancement processing, carrying out the multi-stage decomposition, generating a first image, carrying out the background separation of the first image, generating a region of interest, and carrying out the morphological filtering and edge extraction; generating a binary mask image and performing normalization processing to obtain a standard image; extracting semantic features of the standard image, performing pixel-level classification on the semantic features to obtain a region segmentation result, calculating morphological features and textural features of an esophageal squamous cell carcinoma region, performing classification to obtain an esophageal squamous cell carcinoma classification result, adding the esophageal squamous cell carcinoma classification result as an input feature into a first classification model, and judging esophageal tissues according to a judgment standard to obtain an esophageal squamous cell carcinoma classification result; generating a first judgment result, performing nonlinear mapping and classification to obtain a second judgment result, and performing fine segmentation to obtain an accurate boundary of the early esophageal squamous carcinoma tissue.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to an early esophageal squamous cell carcinoma detection method and system based on deep learning. Background Art

[0002] Esophageal cancer is one of the malignant tumors with high morbidity and mortality worldwide. Among them, esophageal squamous cell carcinoma is the most common type of esophageal cancer, accounting for about 90% of the total number of esophageal cancers. The discovery and diagnosis of early esophageal squamous cell carcinoma is crucial to improving patient survival and prognosis. However, the symptoms of early esophageal squamous cell carcinoma are not obvious and are easily confused with other benign diseases, resulting in frequent misdiagnosis and missed diagnosis. Traditional diagnostic methods for esophageal squamous cell carcinoma mainly rely on endoscopic examination and tissue biopsy, which are somewhat traumatic and subjective, and it is difficult to achieve accurate positioning and quantitative analysis of early lesions.

[0003] In recent years, with the rapid development of medical imaging technology and artificial intelligence, medical image analysis methods based on deep learning have received increasing attention. However, most studies focus on the overall diagnosis of esophageal cancer, and the precise detection and localization of early esophageal squamous cell carcinoma remain challenging.

[0004] Therefore, a solution is urgently needed to solve the problems existing in the prior art. Summary of the invention

[0005] The embodiments of the present invention provide a method and system for detecting early esophageal squamous cell carcinoma based on deep learning, which can at least solve some of the problems existing in the prior art.

[0006] A first aspect of an embodiment of the present invention provides a method for detecting early esophageal squamous cell carcinoma based on deep learning, comprising:

[0007] Acquire an esophageal endoscope image, enhance the esophageal endoscope image by using an adaptive histogram equalization algorithm, perform multi-level decomposition in combination with a multi-scale decomposition method based on wavelet transform to generate a first image, perform background separation on the first image according to an adaptive threshold segmentation algorithm, generate a region of interest and perform morphological filtering and edge extraction, generate a binary mask image and perform normalization processing to obtain a standard image;

[0008] The standard image is added to a pre-trained cascade deep learning model, the semantic features of the standard image are extracted by the residual network in the first deep learning sub-model in the cascade deep learning model, the semantic features are classified at the pixel level by the fully convolutional network structure in the second deep learning sub-model to obtain a regional segmentation result, and based on the regional segmentation result, the morphological features and texture features of the esophageal squamous cell carcinoma region are calculated, and classification is performed in combination with a decision tree model to obtain a grading result of the esophageal squamous cell carcinoma;

[0009] The esophageal squamous cell carcinoma grading result is added as an input feature to the first classification model, and the esophageal tissue is judged according to a preset judgment standard in combination with the area and position of the esophageal squamous cell carcinoma area to generate a first judgment result. According to the first judgment result, in combination with a preset second classification model, the texture features and color features of the esophageal squamous cell carcinoma area are nonlinearly mapped and classified in combination with a mirror set function to obtain a second judgment result, and the esophageal squamous cell carcinoma area is finely segmented in combination with a multi-instance learning algorithm to obtain the precise boundary of the early esophageal squamous cell carcinoma tissue.

[0010] In an optional embodiment,

[0011] Acquire an esophageal endoscope image, enhance the esophageal endoscope image by using an adaptive histogram equalization algorithm, perform multi-level decomposition in combination with a multi-scale decomposition method based on wavelet transform, generate a first image, perform background separation on the first image according to an adaptive threshold segmentation algorithm, generate a region of interest and perform morphological filtering and edge extraction, generate a binary mask image and perform normalization processing, and obtain a standard image including:

[0012] Acquire an esophageal endoscopic image and divide the esophageal endoscopic image into a plurality of identical sub-regions, wherein each sub-region does not overlap with each other, perform histogram equalization processing on each sub-region and fuse the sub-regions after the equalization processing in combination with a bilinear interpolation algorithm to obtain an enhanced image;

[0013] For the enhanced image, multi-scale decomposition is performed on the enhanced image through multi-level wavelet transform, after each level of wavelet decomposition, a low-frequency approximate sub-band and multiple high-frequency detail sub-bands are generated, soft threshold denoising is performed on the high-frequency detail sub-band, noise in the high-frequency detail sub-band is removed and wavelet reconstruction is performed to obtain the first image;

[0014] For the first image, a Gaussian filtering smoothing operation is performed using an adaptive threshold segmentation algorithm, and the grayscale standard deviation and the global standard deviation of the first image are calculated simultaneously. A segmentation threshold is calculated based on the grayscale standard deviation and the global standard deviation, and the image is binarized based on the segmentation threshold to obtain a binary mask image of the inner wall of the esophagus. The binary mask image is smoothed using morphological opening and closing operations, and the maximum connected area is extracted as the region of interest of the inner wall of the esophagus. The inner wall region of the esophagus in the original image is extracted based on the mask corresponding to the region of interest and normalized to obtain the standard image.

[0015] In an optional embodiment,

[0016] A Gaussian filter smoothing operation is performed by using an adaptive threshold segmentation algorithm, and the grayscale standard deviation and the global standard deviation of the first image are calculated at the same time. The segmentation threshold is calculated based on the grayscale standard deviation and the global standard deviation as shown in the following formula:

[0017]

[0018] Where T represents the adaptive threshold, I s represents the image smoothed by adaptive Gaussian filtering, w represents the position weight coefficient, c represents the first control parameter, α represents the second control parameter, σ g represents the global standard deviation of the image, and σ0 represents the grayscale standard deviation of the image.

[0019] In an optional embodiment,

[0020] The standard image is added to a pre-trained cascade deep learning model, the semantic features of the standard image are extracted by the residual network in the first deep learning sub-model in the cascade deep learning model, the semantic features are classified at the pixel level by the fully convolutional network structure in the second deep learning sub-model to obtain a regional segmentation result, based on the regional segmentation result, the morphological features and texture features of the esophageal squamous cell carcinoma region are calculated, and the classification is performed in combination with the decision tree model to obtain the esophageal squamous cell carcinoma grading result including:

[0021] Acquire the standard image and add it to a pre-trained cascade deep learning model, and generate a semantic feature map corresponding to the standard image through forward propagation using a residual network in a first deep learning sub-model in the cascade deep learning model;

[0022] For the semantic feature map, forward propagation is performed through the fully convolutional network structure in the second deep learning sub-model in the cascaded deep learning model, the semantic feature map is downsampled and feature extracted through the encoder, and the decoder restores the spatial resolution of the semantic feature map through upsampling and jump connection, and the classification probability of each pixel belonging to different categories is calculated in combination with the activation function, and a category label is assigned to each pixel according to the classification probability to obtain a pixel-level classification result, and the semantic feature map is regionally segmented based on the pixel-level classification result to generate a segmentation mask of the esophageal squamous cell carcinoma region and a regional segmentation result;

[0023] Based on the segmentation mask, the morphological features of the esophageal squamous cell carcinoma area are extracted, and the texture features of the esophageal squamous cell carcinoma area are calculated in combination with the gray level co-occurrence matrix. The morphological features and the texture features are combined to generate a regional feature vector. The regional feature vector is added to a pre-trained decision tree model and the esophageal squamous cell carcinoma area is divided into different grades. For each standard image, the corresponding esophageal squamous cell carcinoma grading result is output.

[0024] In an optional embodiment,

[0025] Combined with the activation function, the classification probability of each pixel belonging to different categories is calculated, and a category label is assigned to each pixel according to the classification probability as shown in the following formula:

[0026] p i =softmax(z i +θ·s i ·sigmoid(W g ·[f i ,s i ]+b g )+β·c i ·h i );

[0027] Among them, pi represents the probability that pixel i belongs to different categories, zi represents the linear transformation part of the input feature of pixel i, θ represents the first hyperparameter weight, and s i represents the intermediate variable, W g represents the weight matrix, f i represents the feature vector of pixel i, b g represents the bias vector, β represents the feature weight, c i represents the conditional random field weight of the i-th pixel, h i Represents the high-order texture features of the i-th pixel.

[0028] In an optional embodiment,

[0029] The esophageal squamous cell carcinoma grading result is added as an input feature to the first classification model, and the esophageal tissue is judged according to the preset judgment criteria in combination with the area and position of the esophageal squamous cell carcinoma area to generate a first judgment result. According to the first judgment result, in combination with the preset second classification model, the texture features and color features of the esophageal squamous cell carcinoma area are nonlinearly mapped and classified in combination with the mirror set function to obtain a second judgment result, and the esophageal squamous cell carcinoma area is finely segmented in combination with the multi-instance learning algorithm to obtain the precise boundary of the early esophageal squamous cell carcinoma tissue, including:

[0030] The esophageal squamous cell carcinoma grading result is added as an input feature to a preset first classification model, the area and centroid position of each sub-region are calculated as geometric features according to the pre-generated segmentation result, and the esophageal tissue is judged in combination with a pre-set judgment standard to determine whether the current esophageal tissue is early esophageal squamous cell carcinoma and generate a first judgment result. If the current esophageal tissue is early esophageal squamous cell carcinoma, the original image corresponding to the current esophageal tissue is added to the second classification model, otherwise, the first judgment result is output;

[0031] For the original image added to the second classification model, the texture features and color features of the esophageal squamous cell carcinoma area are extracted by the mirror set function, the extracted features are mapped to the high-dimensional feature space by nonlinear mapping, and the disease is classified in combination with the second classifier to obtain the second judgment result. If the current esophageal tissue is judged to be early esophageal squamous cell carcinoma, the image corresponding to the current esophageal tissue is inferred by a multi-instance learning algorithm, the probability of each pixel belonging to a different category is updated and a secondary segmentation mask is generated, and the esophageal squamous cell carcinoma area and the corresponding precise boundary are determined based on the secondary segmentation mask.

[0032] In an optional embodiment,

[0033] The texture features and color features of the esophageal squamous cell carcinoma area are extracted by the mirror set function, and the extracted features are mapped to the high-dimensional feature space by nonlinear mapping as shown in the following formula:

[0034]

[0035] Among them, φ(x) represents the high-dimensional features after mapping, μ j represents the weight coefficient of the jth kernel function, k j (x,x e ) represents the jth kernel function, which is used to measure the original feature x and the sample point x in the mirror set. e similarity, m represents the number of kernel functions, n represents the number of basis functions, δ l represents the weight coefficient of the lth basis function, ψ l () represents the lth basis function.

[0036] A second aspect of an embodiment of the present invention provides an early esophageal squamous cell carcinoma detection system based on deep learning, comprising:

[0037] The first unit is used to acquire an esophageal endoscopic image, enhance the esophageal endoscopic image by an adaptive histogram equalization algorithm, perform multi-level decomposition in combination with a multi-scale decomposition method based on wavelet transform to generate a first image, perform background separation on the first image according to an adaptive threshold segmentation algorithm, generate a region of interest and perform morphological filtering and edge extraction, generate a binary mask image and perform normalization processing to obtain a standard image;

[0038] The second unit is used to add the standard image to a pre-trained cascade deep learning model, extract the semantic features of the standard image through the residual network in the first deep learning sub-model in the cascade deep learning model, and perform pixel-level classification on the semantic features through the fully convolutional network structure in the second deep learning sub-model to obtain a regional segmentation result, calculate the morphological features and texture features of the esophageal squamous cell carcinoma region based on the regional segmentation result, and perform classification in combination with a decision tree model to obtain a grading result of the esophageal squamous cell carcinoma;

[0039] The third unit is used to add the esophageal squamous cell carcinoma grading result as an input feature to the first classification model, and judge the esophageal tissue according to a preset judgment standard in combination with the area and position of the esophageal squamous cell carcinoma area to generate a first judgment result; based on the first judgment result, in combination with a preset second classification model, the texture features and color features of the esophageal squamous cell carcinoma area are nonlinearly mapped and classified in combination with a mirror set function to obtain a second judgment result; and the esophageal squamous cell carcinoma area is refinedly segmented in combination with a multi-instance learning algorithm to obtain the precise boundary of the early esophageal squamous cell carcinoma tissue.

[0040] According to a third aspect of the embodiments of the present invention,

[0041] An electronic device is provided, comprising:

[0042] processor;

[0043] a memory for storing processor-executable instructions;

[0044] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0045] According to a fourth aspect of the embodiments of the present invention,

[0046] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0047] In the present invention, background separation is performed by an adaptive threshold segmentation algorithm, the region of interest is extracted, and morphological filtering and edge extraction are performed to obtain a binary mask image. A series of preprocessing steps can effectively remove noise and interference in the image, highlight the characteristics of the esophageal squamous cell carcinoma region, and lay a good foundation for subsequent classification and segmentation. A cascade deep learning model is introduced, and feature extraction and pixel-level classification are realized through two sub-models, so that accurate segmentation of the esophageal squamous cell carcinoma region is realized, and important regional information is provided for subsequent graded diagnosis. After obtaining the segmentation result of the esophageal squamous cell carcinoma region, the morphological features and texture features of the region are extracted, and classified in combination with the decision tree model to obtain the graded result of the esophageal squamous cell carcinoma. These features can be effectively used for classification to obtain the severity and grade information of the esophageal squamous cell carcinoma, which provides an important reference for clinical diagnosis and treatment. Multi-stage judgment can gradually narrow the diagnosis range and improve the accuracy and reliability of diagnosis. In summary, this embodiment comprehensively utilizes multiple technologies to form a comprehensive and efficient diagnosis system, realizes accurate segmentation and graded diagnosis of the esophageal squamous cell carcinoma region, can adapt to different data sources and application requirements, and has strong practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is a flow chart of a method for detecting early esophageal squamous cell carcinoma based on deep learning according to an embodiment of the present invention;

[0049] Figure 2 This is a schematic diagram of the structure of an early esophageal squamous cell carcinoma detection system based on deep learning according to an embodiment of the present invention. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0052] Figure 1 FIG. 1 is a flow chart of a method for detecting early esophageal squamous cell carcinoma based on deep learning according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0053] S1. Acquire an esophageal endoscope image, enhance the esophageal endoscope image by using an adaptive histogram equalization algorithm, perform multi-level decomposition in combination with a multi-scale decomposition method based on wavelet transform to generate a first image, perform background separation on the first image according to an adaptive threshold segmentation algorithm, generate a region of interest and perform morphological filtering and edge extraction, generate a binary mask image and perform normalization processing to obtain a standard image;

[0054] The adaptive histogram equalization algorithm is an image enhancement technology used to improve the contrast and visibility of an image. The adaptive threshold segmentation algorithm is an image segmentation technology used to segment an image into two parts, a foreground and a background. The morphological filtering is an image processing technology based on mathematical morphology theory. By performing a series of morphological operations on the image, such as corrosion, expansion, opening operations, closing operations, etc., the image features are extracted, noise is removed, and defects are repaired. The binary mask image is a special binary image used to represent the position and shape of the target area.

[0055] In an optional embodiment,

[0056] Acquire an esophageal endoscope image, enhance the esophageal endoscope image by using an adaptive histogram equalization algorithm, perform multi-level decomposition in combination with a multi-scale decomposition method based on wavelet transform, generate a first image, perform background separation on the first image according to an adaptive threshold segmentation algorithm, generate a region of interest and perform morphological filtering and edge extraction, generate a binary mask image and perform normalization processing, and obtain a standard image including:

[0057] Acquire an esophageal endoscopic image and divide the esophageal endoscopic image into a plurality of identical sub-regions, wherein each sub-region does not overlap with each other, perform histogram equalization processing on each sub-region and fuse the sub-regions after the equalization processing in combination with a bilinear interpolation algorithm to obtain an enhanced image;

[0058] For the enhanced image, multi-scale decomposition is performed on the enhanced image through multi-level wavelet transform, after each level of wavelet decomposition, a low-frequency approximate sub-band and multiple high-frequency detail sub-bands are generated, soft threshold denoising is performed on the high-frequency detail sub-band, noise in the high-frequency detail sub-band is removed and wavelet reconstruction is performed to obtain the first image;

[0059] For the first image, a Gaussian filtering smoothing operation is performed using an adaptive threshold segmentation algorithm, and the grayscale standard deviation and the global standard deviation of the first image are calculated simultaneously. A segmentation threshold is calculated based on the grayscale standard deviation and the global standard deviation, and the image is binarized based on the segmentation threshold to obtain a binary mask image of the inner wall of the esophagus. The binary mask image is smoothed using morphological opening and closing operations, and the maximum connected area is extracted as the region of interest of the inner wall of the esophagus. The inner wall region of the esophagus in the original image is extracted based on the mask corresponding to the region of interest and normalized to obtain the standard image.

[0060] The bilinear interpolation is an interpolation method commonly used in image scaling, which generates new pixel values ​​by interpolating the pixel values ​​in the image. The multi-level wavelet transform is a method of decomposing a signal or image into different frequency bands. The low-frequency approximation subband contains the main features and overall structure of the signal or image, and the high-frequency detail subband captures the details and edge information of the signal or image. The soft threshold denoising is a denoising technology based on wavelet transform. By processing the coefficients after wavelet transform, the coefficients less than the threshold are set to zero, and the coefficients greater than the threshold are reduced by a certain proportion, so as to achieve the purpose of denoising. The Gaussian filtering is a technology for smoothing images. The image is convolved with a Gaussian kernel through a convolution operation to achieve the effects of denoising and smoothing. The wavelet reconstruction is the inverse process of the wavelet transform. The binary segmentation is an image processing technology that divides pixels into foreground and background by converting a grayscale image into a binary image.

[0061] Obtain an esophageal endoscope image and preprocess the image. Divide the preprocessed esophageal endoscope image into multiple sub-regions of the same size and non-overlapping, determine the size and number of the sub-regions so that each sub-region contains sufficient local information, and divide the image into regular rectangular sub-regions from left to right and from top to bottom to ensure that there is no overlap between the sub-regions. Each pixel belongs to only one sub-region. Calculate the grayscale histogram of each sub-region, equalize the grayscale histogram, stretch the grayscale dynamic range, map and adjust the pixel grayscale values ​​in the sub-region according to the equalized grayscale histogram, determine the position of each pixel in the enhanced image in the original sub-region, and calculate its grayscale value according to the position of the pixel in the original sub-region through a bilinear interpolation algorithm. The bilinear interpolation algorithm obtains a smoothly transitioned grayscale value by weighted averaging the corresponding pixel grayscale values ​​of the four surrounding sub-regions. Repeat the calculation until all pixels in the enhanced image are processed;

[0062] Select appropriate wavelet basis functions and decomposition layers, perform wavelet decomposition on the enhanced image, obtain a low-frequency approximate subband and multiple high-frequency detail subbands after each decomposition, perform soft threshold denoising on the high-frequency detail subband to remove noise interference, perform wavelet reconstruction on the denoised high-frequency detail subband and low-frequency approximate subband to obtain a denoised first image, perform Gaussian filtering and smoothing on the first image to remove subtle noise and details, calculate the grayscale standard deviation and global standard deviation of the first image, adaptively calculate the segmentation threshold based on the grayscale standard deviation and the global standard deviation, use the segmentation threshold to perform binarization segmentation on the first image, and obtain a binary mask image of the inner wall of the esophagus;

[0063] Perform morphological opening operation on the binary mask image to remove small noise areas, perform morphological closing operation on the image after opening operation to fill small hole areas, extract the maximum connected area in the image after closing operation as the region of interest of the esophageal inner wall, perform pixel-by-pixel AND operation on the mask of the region of interest and the original image, retain pixels with mask value of 1, remove pixels with mask value of 0, obtain the esophageal inner wall region image, calculate the maximum and minimum grayscale values ​​of the esophageal inner wall region image, map the grayscale value to the standard range of [0, 1] through linear stretching, and obtain the grayscale normalized esophageal inner wall region image, that is, the standard image.

[0064] In this embodiment, the esophageal endoscope image is divided into multiple identical sub-regions, and each sub-region is subjected to histogram equalization processing, which can enhance local contrast and highlight local detail information. Multi-level wavelet transform and soft threshold denoising are performed on the enhanced image, which can remove noise in the image at multiple scales and retain important structure and detail information of the image. The binary mask image is smoothed by morphological opening and closing operations, which can remove small noise areas and holes in the segmentation result, improve the accuracy and robustness of the segmentation result, extract the region of interest of the esophageal inner wall, remove irrelevant background and interference areas in the image, and focus subsequent analysis and processing on the most critical esophageal inner wall area, thereby improving computational efficiency and accuracy. In summary, this embodiment realizes enhancement, denoising, segmentation, extraction and standardization of esophageal endoscope images, greatly improves the quality and availability of images, provides a reliable image basis for subsequent diagnosis and treatment of esophageal diseases, can effectively solve various challenges in esophageal endoscope image processing, and has strong practicality and generalizability.

[0065] In an optional embodiment,

[0066] A Gaussian filter smoothing operation is performed by using an adaptive threshold segmentation algorithm, and the grayscale standard deviation and the global standard deviation of the first image are calculated at the same time. The segmentation threshold is calculated based on the grayscale standard deviation and the global standard deviation as shown in the following formula:

[0067]

[0068] Where T represents the adaptive threshold, I s represents the image smoothed by adaptive Gaussian filtering, w represents the position weight coefficient, c represents the first control parameter, α represents the second control parameter, σ g represents the global standard deviation of the image, and σ0 represents the grayscale standard deviation of the image.

[0069] In this embodiment, by calculating the grayscale standard deviation and the global standard deviation of the image, the segmentation threshold can be adaptively adjusted according to the local and global statistical characteristics of the image, and different thresholds are used for segmentation, thereby improving the accuracy and robustness of the segmentation. By introducing the global standard deviation and the grayscale standard deviation, the calculation of the segmentation threshold takes into account the overall statistical characteristics and local grayscale variation characteristics of the image, which helps to adapt to different lighting conditions, contrasts and noise levels of the image. It has strong robustness and can process esophageal endoscopy images of various qualities and imaging conditions. By adjusting the control parameters, the sensitivity of the segmentation threshold to global and local information can be controlled. The segmentation threshold can be flexibly adjusted according to different application requirements and image characteristics to obtain the best segmentation effect. In summary, this embodiment can effectively process esophageal endoscopy images and provide reliable segmentation results for subsequent analysis and diagnosis.

[0070] S2. adding the standard image to a pre-trained cascade deep learning model, extracting semantic features of the standard image through the residual network in the first deep learning sub-model in the cascade deep learning model, performing pixel-level classification on the semantic features through the fully convolutional network structure in the second deep learning sub-model to obtain a regional segmentation result, calculating the morphological features and texture features of the esophageal squamous cell carcinoma region based on the regional segmentation result, and performing classification in combination with a decision tree model to obtain a grading result of the esophageal squamous cell carcinoma;

[0071] The cascaded deep learning model is a strategy that cascades multiple deep learning models to gradually refine the results in stages. The residual network is an architecture that solves the difficulties in training deep neural networks. The semantic features are high-level, meaningful information in images or signals. The fully convolutional network is a neural network architecture specifically used for image segmentation. The pixel-level classification is a technique that classifies each pixel into a specific category, which is commonly used in image segmentation tasks. The morphological features describe the shape and structural characteristics of objects in the image. The decision tree model is a supervised learning algorithm suitable for classification and regression tasks.

[0072] In an optional embodiment,

[0073] The standard image is added to a pre-trained cascade deep learning model, the semantic features of the standard image are extracted by the residual network in the first deep learning sub-model in the cascade deep learning model, the semantic features are classified at the pixel level by the fully convolutional network structure in the second deep learning sub-model to obtain a regional segmentation result, based on the regional segmentation result, the morphological features and texture features of the esophageal squamous cell carcinoma region are calculated, and the classification is performed in combination with the decision tree model to obtain the esophageal squamous cell carcinoma grading result including:

[0074] Acquire the standard image and add it to a pre-trained cascade deep learning model, and generate a semantic feature map corresponding to the standard image through forward propagation using a residual network in a first deep learning sub-model in the cascade deep learning model;

[0075] For the semantic feature map, forward propagation is performed through the fully convolutional network structure in the second deep learning sub-model in the cascaded deep learning model, the semantic feature map is downsampled and feature extracted through the encoder, and the decoder restores the spatial resolution of the semantic feature map through upsampling and jump connection, and the classification probability of each pixel belonging to different categories is calculated in combination with the activation function, and a category label is assigned to each pixel according to the classification probability to obtain a pixel-level classification result, and the semantic feature map is regionally segmented based on the pixel-level classification result to generate a segmentation mask of the esophageal squamous cell carcinoma region and a regional segmentation result;

[0076] Based on the segmentation mask, the morphological features of the esophageal squamous cell carcinoma area are extracted, and the texture features of the esophageal squamous cell carcinoma area are calculated in combination with the gray level co-occurrence matrix. The morphological features and the texture features are combined to generate a regional feature vector. The regional feature vector is added to a pre-trained decision tree model and the esophageal squamous cell carcinoma area is divided into different grades. For each standard image, the corresponding esophageal squamous cell carcinoma grading result is output.

[0077] The semantic feature map is a high-level feature map extracted during image processing, which represents the semantic information in the image. The encoder is a neural network structure used to compress input data (such as images and text) into a low-dimensional feature representation. The decoder is a neural network structure used to restore the low-dimensional feature representation to a high-dimensional form of the original data. The category label is a label used to identify different categories in classification or segmentation tasks. The segmentation mask is an image or matrix in which each pixel value represents the category to which the pixel belongs. The grayscale co-occurrence matrix is ​​a method for statistically analyzing image texture features, which describes the joint distribution of grayscale values ​​in an image and captures the texture information of the image by calculating the co-occurrence relationship between pixel pairs.

[0078] Through image processing, a standard image of the esophageal inner wall area is obtained, and the standard image is input into a pre-trained cascade deep learning model as the input data of the model. A semantic feature map is generated through the first deep learning sub-model of the cascade deep learning model. The standard image is forward propagated in the residual network, and high-level semantic features of the image are extracted through multiple convolutional layers and residual blocks. The output of the residual network is the semantic feature map corresponding to the standard image, which contains high-level semantic information of the image.

[0079] The second deep learning sub-model adopts a fully convolutional network (FCN) structure, which includes an encoder and a decoder. The semantic feature map is downsampled and feature extracted in the encoder. The spatial resolution of the feature map is gradually reduced through multiple convolutional layers and pooling layers, while the number of feature channels is increased. The output of the encoder is upsampled and feature restored through the decoder. The spatial resolution of the feature map is gradually increased through multiple transposed convolutional layers and jump connections, and restored to the same size as the input image. The output of the decoder calculates the classification probability of each pixel belonging to different categories (such as esophageal squamous cell carcinoma area and normal area) through an activation function (such as Softmax). A category label is assigned to each pixel according to the classification probability to obtain a pixel-level classification result. Based on the pixel-level classification result, the semantic feature map is segmented to generate a segmentation mask and a regional segmentation result of the esophageal squamous cell carcinoma area.

[0080] Based on the segmentation mask, the morphological features of the esophageal squamous cell carcinoma region, such as area, perimeter, circularity, compactness, etc., are extracted. The gray-level co-occurrence matrix (GLCM) of the esophageal squamous cell carcinoma region is calculated, and the texture features, such as contrast, correlation, energy, entropy, etc., are extracted. The morphological features and texture features are combined to generate the feature vector of the esophageal squamous cell carcinoma region.

[0081] The feature vector of the esophageal squamous cell carcinoma area is added to the pre-trained decision tree model. The decision tree model divides the esophageal squamous cell carcinoma area into different levels, such as low level, intermediate level, high level, etc., according to the different values ​​of the feature vector. For each standard image, the decision tree model outputs the corresponding esophageal squamous cell carcinoma grading result. For each input standard image, the cascaded deep learning model and the decision tree model work together to generate the corresponding esophageal squamous cell carcinoma grading result.

[0082] In this embodiment, the residual network and the fully convolutional network in the cascaded deep learning model are used to realize the automatic segmentation of the esophageal squamous cell carcinoma area. The residual network extracts high-level semantic features of the image, and the fully convolutional network realizes pixel-level classification and regional segmentation, which greatly improves the efficiency and accuracy of segmentation and reduces the workload of doctors. The morphological features characterize the geometric shape and size of the region, and the texture features characterize the spatial distribution pattern of pixels inside the region, which describes the characteristics of the esophageal squamous cell carcinoma area from different angles and provides rich information for subsequent graded diagnosis. The feature vector of the esophageal squamous cell carcinoma area is input into the decision tree model to realize the graded diagnosis of esophageal squamous cell carcinoma, which reflects the severity and development stage of esophageal squamous cell carcinoma, is more objective and accurate, and reduces the influence of subjective factors. In summary, this embodiment realizes the automatic segmentation and graded diagnosis of the esophageal squamous cell carcinoma area, greatly improves the efficiency, accuracy and consistency of diagnosis, and provides strong support for the diagnosis and treatment of esophageal squamous cell carcinoma.

[0083] In an optional embodiment,

[0084] Combined with the activation function, the classification probability of each pixel belonging to different categories is calculated, and a category label is assigned to each pixel according to the classification probability as shown in the following formula:

[0085] p i =softmax(z i +θ·s i ·sigmoid(W g ·[f i ,s i ]+b g )+β·c i ·h i );

[0086] Among them, p i represents the probability that pixel i belongs to different categories, z i represents the linear transformation part of the input feature of pixel i, θ represents the first hyperparameter weight, and s i represents the intermediate variable, W g represents the weight matrix, f i represents the feature vector of pixel i, b g represents the bias vector, β represents the feature weight, c i represents the conditional random field weight of the i-th pixel, h i Represents the high-order texture features of the i-th pixel.

[0087] In this embodiment, a variety of different feature information is considered, including pixel input features, intermediate variables, feature vectors, conditional random field weights and high-order texture features. Through linear and nonlinear transformations and weighted fusion, a more comprehensive and accurate pixel representation is obtained, thereby improving the accuracy of classification. In the calculation of classification probability, the idea of ​​conditional random fields is introduced, and the spatial dependency between pixels can be considered. The mutual influence and constraint relationship between pixels can be modeled, so that the classification labels of adjacent pixels are more consistent and smooth, thereby improving the spatial continuity of the segmentation results. In the calculation formula of classification probability, multiple learnable parameters are introduced, such as weight matrix, bias vector and hyperparameter weight, thereby enabling the classification model to adaptively adjust the importance and contribution of different features, thereby improving the generalization ability of classification. In summary, this embodiment improves the accuracy and spatial continuity of pixel classification, provides important pixel-level information for subsequent regional segmentation and diagnosis of esophageal squamous cell carcinoma, and realizes fine classification at the pixel level.

[0088] S3. Add the esophageal squamous cell carcinoma grading result as an input feature to the first classification model, combine the area and position of the esophageal squamous cell carcinoma area, and judge the esophageal tissue according to the preset judgment criteria to generate a first judgment result. Based on the first judgment result, combined with the preset second classification model, the texture features and color features of the esophageal squamous cell carcinoma area are nonlinearly mapped and classified in combination with the mirror set function to obtain the second judgment result, and the esophageal squamous cell carcinoma area is refinedly segmented in combination with the multi-instance learning algorithm to obtain the precise boundary of the early esophageal squamous cell carcinoma tissue.

[0089] The mirror set function generally refers to a function generated by performing a mirror inversion operation on a set in the field of mathematics or image processing. The nonlinear mapping is a process of converting input data into different outputs through a nonlinear function. The fine segmentation is a high-precision image segmentation technology that aims to segment an image into more detailed and accurate regions. The precise boundary refers to the fact that in the image segmentation task, the boundary of the segmented region is highly consistent with the boundary of the real object.

[0090] In an optional embodiment,

[0091] The esophageal squamous cell carcinoma grading result is added as an input feature to the first classification model, and the esophageal tissue is judged according to the preset judgment criteria in combination with the area and position of the esophageal squamous cell carcinoma area to generate a first judgment result. According to the first judgment result, in combination with the preset second classification model, the texture features and color features of the esophageal squamous cell carcinoma area are nonlinearly mapped and classified in combination with the mirror set function to obtain a second judgment result, and the esophageal squamous cell carcinoma area is finely segmented in combination with the multi-instance learning algorithm to obtain the precise boundary of the early esophageal squamous cell carcinoma tissue, including:

[0092] The esophageal squamous cell carcinoma grading result is added as an input feature to a preset first classification model, the area and centroid position of each sub-region are calculated as geometric features according to the pre-generated segmentation result, and the esophageal tissue is judged in combination with a pre-set judgment standard to determine whether the current esophageal tissue is early esophageal squamous cell carcinoma and generate a first judgment result. If the current esophageal tissue is early esophageal squamous cell carcinoma, the original image corresponding to the current esophageal tissue is added to the second classification model, otherwise, the first judgment result is output;

[0093] For the original image added to the second classification model, the texture features and color features of the esophageal squamous cell carcinoma area are extracted by the mirror set function, the extracted features are mapped to the high-dimensional feature space by nonlinear mapping, and the disease is classified in combination with the second classifier to obtain the second judgment result. If the current esophageal tissue is judged to be early esophageal squamous cell carcinoma, the image corresponding to the current esophageal tissue is inferred by a multi-instance learning algorithm, the probability of each pixel belonging to a different category is updated and a secondary segmentation mask is generated, and the esophageal squamous cell carcinoma area and the corresponding precise boundary are determined based on the secondary segmentation mask.

[0094] The centroid position is a geometric concept, which represents the center point of a shape or object, that is, the average position of all points. In image processing, the centroid is usually used to describe the center position of a segmented area or object. The esophageal tissue refers to the biological tissue that constitutes the esophagus. The high-dimensional feature space refers to representing data in a high-dimensional space by extracting high-dimensional features (such as feature vectors in deep learning). The secondary segmentation mask is a mask obtained by further refining the segmentation based on the initial segmentation result.

[0095] Obtain the esophageal squamous cell carcinoma grading result corresponding to each standard image obtained in advance, use the grading result as an input feature, add it to the pre-set first classification model for subsequent early esophageal squamous cell carcinoma determination, divide the esophageal tissue into multiple sub-regions according to the pre-generated esophageal squamous cell carcinoma region segmentation mask, calculate the area and centroid position of each sub-region as the geometric features of the region, and use the geometric features of each sub-region together with the esophageal squamous cell carcinoma grading result as the input features of the first classification model;

[0096] According to the output result of the first classification model, combined with the preset judgment criteria, the current esophageal tissue is judged, wherein the judgment criteria may include multiple factors such as esophageal squamous cell carcinoma grade, sub-region area, centroid position, etc. If the current esophageal tissue meets the judgment criteria of early esophageal squamous cell carcinoma, a first judgment result is generated, indicating that the current tissue is early esophageal squamous cell carcinoma; if the current esophageal tissue does not meet the judgment criteria of early esophageal squamous cell carcinoma, the first judgment result is directly output, indicating that the current tissue is not early esophageal squamous cell carcinoma;

[0097] If the first judgment result indicates that the current esophageal tissue is early esophageal squamous cell carcinoma, the corresponding original image is added to the second classification model. For the original image added to the second classification model, the texture features and color features of the esophageal squamous cell carcinoma region are extracted through the mirror set function. The mirror set function can generate a set of mirror features according to the local texture and color distribution of the image to characterize the texture and color characteristics of the esophageal squamous cell carcinoma region. The extracted texture features and color features are mapped to a high-dimensional feature space through nonlinear mapping. The features mapped to the high-dimensional feature space are input into the second classifier. Through the training and prediction of the second classifier, the second judgment result of the current esophageal tissue is obtained to determine whether it is early esophageal squamous cell carcinoma.

[0098] If the second judgment result indicates that the current esophageal tissue is early esophageal squamous cell carcinoma, it is further inferred and segmented through a multi-instance learning algorithm. The local and global features of the esophageal squamous cell carcinoma area are used to update the probability that each pixel belongs to different categories. Through iterative optimization of the multi-instance learning algorithm, a secondary segmentation mask is generated to accurately locate the boundary of the esophageal squamous cell carcinoma area. The secondary segmentation mask generated by the multi-instance learning algorithm is used to accurately determine the boundary of the esophageal squamous cell carcinoma area, extract the precise boundary of the esophageal squamous cell carcinoma area, and use the esophageal squamous cell carcinoma area and the corresponding precise boundary as the final output result to provide accurate location information for subsequent treatment and intervention.

[0099] In this embodiment, according to the pre-generated segmentation results, the geometric features such as the area and centroid position of each sub-region are calculated, which reflects the morphology and position information of the esophageal squamous cell carcinoma area, can effectively distinguish normal tissue from abnormal tissue, and provide an important reference for the determination of early esophageal squamous cell carcinoma. Two classification models are used to perform staged determination of esophageal tissue, which can gradually narrow the scope of determination and improve the accuracy and reliability of determination. Through nonlinear mapping, the extracted texture and color features are mapped to a high-dimensional feature space, which can better characterize the nonlinear relationship between features and enhance the discrimination ability of features. Further inference and segmentation are performed through a multi-instance learning algorithm, which can effectively deal with the uncertainty and diversity of the esophageal squamous cell carcinoma area and improve the accuracy of segmentation. In summary, this embodiment effectively improves the accuracy, precision and reliability of the diagnosis of early esophageal squamous cell carcinoma, and provides important technical support for clinical diagnosis and treatment.

[0100] In an optional embodiment,

[0101] The texture features and color features of the esophageal squamous cell carcinoma area are extracted by the mirror set function, and the extracted features are mapped to the high-dimensional feature space by nonlinear mapping as shown in the following formula:

[0102]

[0103] Among them, φ(x) represents the high-dimensional features after mapping, μ j represents the weight coefficient of the jth kernel function, k j (x,x e ) represents the jth kernel function, which is used to measure the original feature x and the sample point x in the mirror set. e similarity, m represents the number of kernel functions, n represents the number of basis functions, δ l represents the weight coefficient of the lth basis function, ψ l () represents the lth basis function.

[0104] In this embodiment, nonlinear mapping is used to map the extracted features to a high-dimensional feature space, which can effectively capture the nonlinear relationship and high-order statistical information between features. The concept of kernel function is introduced in the nonlinear mapping to measure the similarity between the original features and the sample points in the mirror set, which can effectively characterize the nonlinear similarity between the features and improve the effect of feature mapping. By introducing basis functions to capture the high-order statistical information and structured representation of the features, the expressive power of the features can be improved, and the complex relationships and dependencies between the features can be captured. In summary, this embodiment makes full use of the advantages of kernel functions and basis functions, and achieves efficient expression and discrimination of features through technologies such as expanding training samples, capturing nonlinear relationships, and extracting high-order statistical information, which lays a good foundation for subsequent classification and judgment, and provides new ideas and methods for the diagnosis and treatment of esophageal squamous cell carcinoma.

[0105] Figure 2 FIG. 1 is a schematic diagram of the structure of an early esophageal squamous cell carcinoma detection system based on deep learning according to an embodiment of the present invention. Figure 2 As shown, the system comprises:

[0106] The first unit is used to acquire an esophageal endoscopic image, enhance the esophageal endoscopic image by an adaptive histogram equalization algorithm, perform multi-level decomposition in combination with a multi-scale decomposition method based on wavelet transform to generate a first image, perform background separation on the first image according to an adaptive threshold segmentation algorithm, generate a region of interest and perform morphological filtering and edge extraction, generate a binary mask image and perform normalization processing to obtain a standard image;

[0107] The second unit is used to add the standard image to a pre-trained cascade deep learning model, extract the semantic features of the standard image through the residual network in the first deep learning sub-model in the cascade deep learning model, and perform pixel-level classification on the semantic features through the fully convolutional network structure in the second deep learning sub-model to obtain a regional segmentation result, calculate the morphological features and texture features of the esophageal squamous cell carcinoma region based on the regional segmentation result, and perform classification in combination with a decision tree model to obtain a grading result of the esophageal squamous cell carcinoma;

[0108] The third unit is used to add the esophageal squamous cell carcinoma grading result as an input feature to the first classification model, and judge the esophageal tissue according to a preset judgment standard in combination with the area and position of the esophageal squamous cell carcinoma area to generate a first judgment result; based on the first judgment result, in combination with a preset second classification model, the texture features and color features of the esophageal squamous cell carcinoma area are nonlinearly mapped and classified in combination with a mirror set function to obtain a second judgment result; and the esophageal squamous cell carcinoma area is refinedly segmented in combination with a multi-instance learning algorithm to obtain the precise boundary of the early esophageal squamous cell carcinoma tissue.

[0109] According to a third aspect of the embodiments of the present invention,

[0110] An electronic device is provided, comprising:

[0111] processor;

[0112] a memory for storing processor-executable instructions;

[0113] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0114] According to a fourth aspect of the embodiments of the present invention,

[0115] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0116] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0117] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting early esophageal squamous cell carcinoma based on deep learning, characterized in that: include: Acquire an esophageal endoscope image, enhance the esophageal endoscope image by using an adaptive histogram equalization algorithm, perform multi-level decomposition in combination with a multi-scale decomposition method based on wavelet transform to generate a first image, perform background separation on the first image according to an adaptive threshold segmentation algorithm, generate a region of interest and perform morphological filtering and edge extraction, generate a binary mask image and perform normalization processing to obtain a standard image; The standard image is added to a pre-trained cascade deep learning model, the semantic features of the standard image are extracted by the residual network in the first deep learning sub-model in the cascade deep learning model, the semantic features are classified at the pixel level by the fully convolutional network structure in the second deep learning sub-model to obtain a regional segmentation result, and based on the regional segmentation result, the morphological features and texture features of the esophageal squamous cell carcinoma region are calculated, and classification is performed in combination with a decision tree model to obtain a grading result of the esophageal squamous cell carcinoma; The esophageal squamous cell carcinoma grading result is added as an input feature to the first classification model, and the esophageal tissue is judged according to a preset judgment standard in combination with the area and position of the esophageal squamous cell carcinoma area to generate a first judgment result. According to the first judgment result, in combination with a preset second classification model, the texture features and color features of the esophageal squamous cell carcinoma area are nonlinearly mapped and classified in combination with a mirror set function to obtain a second judgment result, and the esophageal squamous cell carcinoma area is finely segmented in combination with a multi-instance learning algorithm to obtain the precise boundary of the early esophageal squamous cell carcinoma tissue.

2. The method according to claim 1, characterized in that Acquire an esophageal endoscope image, enhance the esophageal endoscope image by using an adaptive histogram equalization algorithm, perform multi-level decomposition in combination with a multi-scale decomposition method based on wavelet transform, generate a first image, perform background separation on the first image according to an adaptive threshold segmentation algorithm, generate a region of interest and perform morphological filtering and edge extraction, generate a binary mask image and perform normalization processing, and obtain a standard image including: Acquire an esophageal endoscopic image and divide the esophageal endoscopic image into a plurality of identical sub-regions, wherein each sub-region does not overlap with each other, perform histogram equalization processing on each sub-region and fuse the sub-regions after the equalization processing in combination with a bilinear interpolation algorithm to obtain an enhanced image; For the enhanced image, multi-scale decomposition is performed on the enhanced image through multi-level wavelet transform, after each level of wavelet decomposition, a low-frequency approximate sub-band and multiple high-frequency detail sub-bands are generated, soft threshold denoising is performed on the high-frequency detail sub-band, noise in the high-frequency detail sub-band is removed and wavelet reconstruction is performed to obtain the first image; For the first image, a Gaussian filtering smoothing operation is performed using an adaptive threshold segmentation algorithm, and the grayscale standard deviation and the global standard deviation of the first image are calculated simultaneously. A segmentation threshold is calculated based on the grayscale standard deviation and the global standard deviation, and the image is binarized based on the segmentation threshold to obtain a binary mask image of the inner wall of the esophagus. The binary mask image is smoothed using morphological opening and closing operations, and the maximum connected area is extracted as the region of interest of the inner wall of the esophagus. The inner wall region of the esophagus in the original image is extracted based on the mask corresponding to the region of interest and normalized to obtain the standard image.

3. The method according to claim 2, characterized in that A Gaussian filter smoothing operation is performed by using an adaptive threshold segmentation algorithm, and the grayscale standard deviation and the global standard deviation of the first image are calculated at the same time. The segmentation threshold is calculated based on the grayscale standard deviation and the global standard deviation as shown in the following formula: Where T represents the adaptive threshold, I s represents the image smoothed by adaptive Gaussian filtering, w represents the position weight coefficient, c represents the first control parameter, α represents the second control parameter, σ g represents the global standard deviation of the image, and σ0 represents the grayscale standard deviation of the image.

4. The method according to claim 1, characterized in that: The standard image is added to a pre-trained cascade deep learning model, the semantic features of the standard image are extracted by the residual network in the first deep learning sub-model in the cascade deep learning model, the semantic features are classified at the pixel level by the fully convolutional network structure in the second deep learning sub-model to obtain a regional segmentation result, based on the regional segmentation result, the morphological features and texture features of the esophageal squamous cell carcinoma region are calculated, and the classification is performed in combination with the decision tree model to obtain the esophageal squamous cell carcinoma grading result including: Acquire the standard image and add it to a pre-trained cascade deep learning model, and generate a semantic feature map corresponding to the standard image through forward propagation using a residual network in a first deep learning sub-model in the cascade deep learning model; For the semantic feature map, forward propagation is performed through the fully convolutional network structure in the second deep learning sub-model in the cascaded deep learning model, the semantic feature map is downsampled and feature extracted through the encoder, and the decoder restores the spatial resolution of the semantic feature map through upsampling and jump connection, and the classification probability of each pixel belonging to different categories is calculated in combination with the activation function, and a category label is assigned to each pixel according to the classification probability to obtain a pixel-level classification result, and the semantic feature map is regionally segmented based on the pixel-level classification result to generate a segmentation mask of the esophageal squamous cell carcinoma region and a regional segmentation result; Based on the segmentation mask, the morphological features of the esophageal squamous cell carcinoma area are extracted, and the texture features of the esophageal squamous cell carcinoma area are calculated in combination with the gray level co-occurrence matrix. The morphological features and the texture features are combined to generate a regional feature vector. The regional feature vector is added to a pre-trained decision tree model and the esophageal squamous cell carcinoma area is divided into different grades. For each standard image, the corresponding esophageal squamous cell carcinoma grading result is output.

5. The method according to claim 4, characterized in that Combined with the activation function, the classification probability of each pixel belonging to different categories is calculated, and a category label is assigned to each pixel according to the classification probability as shown in the following formula: p i =softmax(z i +θ·s i ·sigmoid(W g ·[f i ,s i ]+b g )+β·c i ·h i ); Among them, pi represents the probability that pixel i belongs to different categories, zi represents the linear transformation part of the input feature of pixel i, θ represents the first hyperparameter weight, and s i represents the intermediate variable, W g represents the weight matrix, fi represents the feature vector of pixel i, bg represents the bias vector, β represents the feature weight, ci represents the conditional random field weight of the i-th pixel, and h i Represents the high-order texture features of the i-th pixel.

6. The method according to claim 1, characterized in that The esophageal squamous cell carcinoma grading result is added as an input feature to the first classification model, and the esophageal tissue is judged according to the preset judgment criteria in combination with the area and position of the esophageal squamous cell carcinoma area to generate a first judgment result. According to the first judgment result, in combination with the preset second classification model, the texture features and color features of the esophageal squamous cell carcinoma area are nonlinearly mapped and classified in combination with the mirror set function to obtain a second judgment result, and the esophageal squamous cell carcinoma area is finely segmented in combination with the multi-instance learning algorithm to obtain the precise boundary of the early esophageal squamous cell carcinoma tissue, including: The esophageal squamous cell carcinoma grading result is added as an input feature to a preset first classification model, the area and centroid position of each sub-region are calculated as geometric features according to the pre-generated segmentation result, and the esophageal tissue is judged in combination with a pre-set judgment standard to determine whether the current esophageal tissue is early esophageal squamous cell carcinoma and generate a first judgment result. If the current esophageal tissue is early esophageal squamous cell carcinoma, the original image corresponding to the current esophageal tissue is added to the second classification model, otherwise, the first judgment result is output; For the original image added to the second classification model, the texture features and color features of the esophageal squamous cell carcinoma area are extracted by the mirror set function, the extracted features are mapped to the high-dimensional feature space by nonlinear mapping, and the disease is classified in combination with the second classifier to obtain the second judgment result. If the current esophageal tissue is judged to be early esophageal squamous cell carcinoma, the image corresponding to the current esophageal tissue is inferred by a multi-instance learning algorithm, the probability of each pixel belonging to a different category is updated and a secondary segmentation mask is generated, and the esophageal squamous cell carcinoma area and the corresponding precise boundary are determined based on the secondary segmentation mask.

7. The method according to claim 6, characterized in that The texture features and color features of the esophageal squamous cell carcinoma area are extracted by the mirror set function, and the extracted features are mapped to the high-dimensional feature space by nonlinear mapping as shown in the following formula: Among them, φ(x) represents the high-dimensional features after mapping, μ j represents the weight coefficient of the jth kernel function, k j (x,x e ) represents the jth kernel function, which is used to measure the original feature x and the sample point x in the mirror set. e similarity, m represents the number of kernel functions, n represents the number of basis functions, δ l represents the weight coefficient of the lth basis function, ψ l () represents the lth basis function.

8. An early esophageal squamous cell carcinoma detection system based on deep learning, used to implement the method described in any one of claims 1 to 7, characterized in that: include: The first unit is used to acquire an esophageal endoscopic image, enhance the esophageal endoscopic image by an adaptive histogram equalization algorithm, perform multi-level decomposition in combination with a multi-scale decomposition method based on wavelet transform to generate a first image, perform background separation on the first image according to an adaptive threshold segmentation algorithm, generate a region of interest and perform morphological filtering and edge extraction, generate a binary mask image and perform normalization processing to obtain a standard image; The second unit is used to add the standard image to a pre-trained cascade deep learning model, extract the semantic features of the standard image through the residual network in the first deep learning sub-model in the cascade deep learning model, and perform pixel-level classification on the semantic features through the fully convolutional network structure in the second deep learning sub-model to obtain a regional segmentation result, calculate the morphological features and texture features of the esophageal squamous cell carcinoma region based on the regional segmentation result, and perform classification in combination with a decision tree model to obtain a grading result of the esophageal squamous cell carcinoma; The third unit is used to add the esophageal squamous cell carcinoma grading result as an input feature to the first classification model, and judge the esophageal tissue according to a preset judgment standard in combination with the area and position of the esophageal squamous cell carcinoma area to generate a first judgment result; based on the first judgment result, in combination with a preset second classification model, the texture features and color features of the esophageal squamous cell carcinoma area are nonlinearly mapped and classified in combination with a mirror set function to obtain a second judgment result; and the esophageal squamous cell carcinoma area is refinedly segmented in combination with a multi-instance learning algorithm to obtain the precise boundary of the early esophageal squamous cell carcinoma tissue.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images