An analysis method, device and storage medium for early gastric cancer and precancerous lesions

Abstract

The application discloses an early gastric cancer and precancerous lesion analysis method and device and a storage medium, acquires a to-be-detected gastroscope image; a target feature extraction deep network is used to extract features of the to-be-detected gastroscope image, and corresponding target feature vectors and target feature maps are obtained; a target diagnosis model is used to diagnose the types of the target feature vectors, and corresponding classification results are obtained; based on the classification results and the target feature maps, a target boundary recognition model is used to mark the boundaries and lesion degrees of the lesion regions, and corresponding target segmentation maps are obtained; and based on the target feature vectors, the classification results, the target segmentation maps and historical data of the to-be-detected gastroscope image, a target risk prediction model is used to obtain risk prediction results and analysis results. Thus, the application can accurately identify the types and lesion progression risks of precancerous lesions, thereby assisting doctors in accurately judging the illness, avoiding underestimating the illness and effectively preventing the occurrence of gastric cancer.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, in particular to an early gastric cancer and precancerous lesion analysis method, device and storage medium. BACKGROUND

[0002] At present, gastric cancer is one of the most common cancers in the world, and the development of gastric cancer is a multi-stage process, which may include different stages of chronic atrophic gastritis, intestinal metaplasia, low-grade intraepithelial neoplasia, and high-grade intraepithelial neoplasia. Among them, the occurrence process of gastric cancer can be: normal gastric mucosa→chronic inflammation→chronic atrophic gastritis→intestinal metaplasia→low-grade intraepithelial neoplasia→high-grade intraepithelial neoplasia→early gastric cancer.

[0003] In the prior art, artificial intelligence assisted digestive endoscopy can be used to identify gastric cancer and determine the depth of gastric cancer infiltration, but it cannot identify or accurately identify the stages of precancerous lesions, which may lead to ineffective prevention of the occurrence of gastric cancer. SUMMARY

[0004] The present application aims to at least solve one of the technical problems in the related art.

[0005] To this end, the present application provides an early gastric cancer and precancerous lesion analysis method, which can accurately identify the type and lesion progression risk of precancerous lesions, thereby assisting physicians in accurately judging the disease, avoiding underestimating the disease, and effectively preventing the occurrence of gastric cancer.

[0006] Another object of the present application is to provide an early gastric cancer and precancerous lesion analysis device.

[0007] To achieve the above-mentioned purpose, the present application provides an early gastric cancer and precancerous lesion analysis method, which comprises:

[0008] Obtaining a to-be-detected gastroscope image;

[0009] Extracting features of the to-be-detected gastroscope image through a target feature extraction deep network to obtain corresponding target feature vectors and target feature maps;

[0010] Diagnosing the type of the target feature vector through a target diagnosis model to obtain a corresponding classification result;

[0011] Based on the classification result and the target feature map, marking the boundary and lesion degree of the lesion area through a target boundary recognition model to obtain a corresponding target segmentation map;

[0012] Based on the target feature vector, the classification result, the target segmentation map and the historical data of the to-be-detected gastroscope image, obtaining a risk prediction result and an analysis result through a target risk prediction model.

[0013] The early gastric cancer and precancerous lesion analysis method of the embodiment of the present application can further have the following additional technical features:

[0014] In an embodiment of the present application, the target feature extraction deep network comprises a first input layer, a convolutional network layer, a noise reduction layer and a first output layer; the feature extraction of the to-be-detected gastroscope image through the target feature extraction deep network to obtain the corresponding target feature vector and target feature map comprises:

[0015] The to-be-detected gastroscope image is transmitted to the convolutional network layer through the first input layer;

[0016] The convolutional network layer extracts features of the to-be-detected gastroscope image through a plurality of convolutional layers and SE modules to obtain a first feature image;

[0017] The noise reduction layer removes noise from the first feature image to obtain a second feature image;

[0018] The first output layer obtains the corresponding target feature vector and target feature map based on the second feature image.

[0019] In an embodiment of the present application, there is a corresponding SE module after each convolutional layer; the convolutional network layer extracts features of the to-be-detected gastroscope image through a plurality of convolutional layers and SE modules to obtain a first feature image, comprising:

[0020] Features of the to-be-detected gastroscope image are extracted through a first convolutional layer to obtain a third feature image;

[0021] Features of the third feature image are processed through a first SE module to obtain a fourth feature image;

[0022] Features of the fourth feature image are extracted through a second convolutional layer to obtain a fifth feature image;

[0023] Features of the fifth feature image are processed through a second SE module to obtain a sixth feature image;

[0024] The above steps are repeated, and the feature image output by the SE module corresponding to the last convolutional layer is determined as the first feature image.

[0025] In an embodiment of the present application, the first SE module comprises a global average pooling layer, a first full connection layer, a ReLU activation function layer, a second full connection layer, a Sigmoid activation function layer and an adaptive weighting layer; features of the third feature image are processed through the first SE module to obtain the fourth feature image, comprising:

[0026] The global average pooling layer globally averages the third feature image to obtain a first feature vector;

[0027] The first fully connected layer performs dimension compression on the first feature vector to obtain a second feature vector;

[0028] The ReLU activation function layer applies the ReLU activation function to the second feature vector to obtain a third feature vector;

[0029] The second fully connected layer performs dimension expansion on the third feature vector to obtain a first weight coefficient of each channel;

[0030] The Sigmoid activation function layer performs Sigmoid activation on the first weight coefficient to obtain a second weight coefficient of each channel;

[0031] The adaptive weighting layer multiplies the second weight coefficient and the third feature image by channel to obtain a fourth feature image.

[0032] In an embodiment of the present application, the target diagnosis model comprises a second input layer, a third fully connected layer, a Dropout layer, a classification layer and a second output layer; the type diagnosis of the target feature vector by the target diagnosis model obtains a corresponding classification result, which comprises:

[0033] The target feature vector is transmitted to the third fully connected layer through the second input layer;

[0034] The third fully connected layer performs feature processing on the target feature vector to obtain a fourth feature vector;

[0035] The fourth feature vector passes through the Dropout layer to obtain a fifth feature vector;

[0036] The classification layer classifies the fifth feature vector by the Softmax function to obtain a probability vector;

[0037] The second output layer determines the class corresponding to the maximum probability value in the probability vector as the classification result and outputs it.

[0038] In an embodiment of the present application, the target boundary recognition model comprises a third input layer, an encoder, a skip connection layer, a decoder, a multi-scale feature fusion layer and a third output layer; the boundary and the lesion degree of the lesion area are marked based on the classification result and the target feature map by the target boundary recognition model to obtain a corresponding target segmentation map, which comprises:

[0039] The classification result and the target feature map are transmitted to the encoder through the third input layer;

[0040] The encoder extracts spatial features of different scales by downsampling the target feature map to obtain a seventh feature image;

[0041] The skip connection layer transmits each layer feature of the encoder to the corresponding layer of the decoder;

[0042] The decoder recovers the spatial resolution of the seventh feature image layer by layer to obtain the first segmentation map corresponding to the target feature map;

[0043] The multi-scale feature fusion layer fuses the first segmentation map to obtain a second segmentation map;

[0044] The third output layer obtains the corresponding target segmentation map based on the classification result and the second segmentation map.

[0045] In an embodiment of the present application, the target risk prediction model comprises a fourth input layer, a time series analysis layer, a risk assessment layer and a fourth output layer; based on the target feature vector, the classification result, the target segmentation map and the historical data of the to-be-detected gastroscope image, the risk prediction result and the analysis result are obtained through the target risk prediction model, comprising:

[0046] Obtain the historical data of the to-be-detected gastroscope image;

[0047] The target feature vector, the classification result, the target segmentation map and the historical data of the to-be-detected gastroscope image are transmitted to the time series analysis layer through the fourth input layer;

[0048] The time series analysis layer obtains the corresponding sixth feature vector by using the LSTM network through the target feature vector, the classification result, the target segmentation map and the historical data of the to-be-detected gastroscope image;

[0049] The risk assessment layer analyzes the sixth feature vector to obtain a predicted risk value;

[0050] The fourth output layer outputs the predicted risk value as a risk prediction result;

[0051] Based on the risk prediction result, the corresponding analysis result is obtained.

[0052] To achieve the above purpose, another aspect of the present application provides an analysis device for early gastric cancer and precancerous lesions, the device comprising:

[0053] An acquisition module for acquiring a to-be-detected gastroscope image;

[0054] An extraction module for extracting features of the to-be-detected gastroscope image through a target feature extraction deep network to obtain a corresponding target feature vector and a target feature map;

[0055] a classification module configured to perform type diagnosis on the target feature vector by using a target diagnosis model to obtain a corresponding classification result;

[0056] a processing module configured to mark a boundary and a lesion degree of a lesion area by using a target boundary recognition model based on the classification result and the target feature map to obtain a corresponding target segmentation map;

[0057] a risk prediction module configured to obtain a risk prediction result and an analysis result by using a target risk prediction model based on the target feature vector, the classification result, the target segmentation map and historical data of the to-be-detected gastroscope image.

[0058] The early gastric cancer and precancerous lesion analysis method, device and storage medium provided by the embodiment of the present application obtain a to-be-detected gastroscope image; a target feature extraction deep network is used to extract features of the to-be-detected gastroscope image to obtain a corresponding target feature vector and target feature map; a target diagnosis model is used to perform type diagnosis on the target feature vector to obtain a corresponding classification result; a target boundary recognition model is used to mark a boundary and a lesion degree of a lesion area based on the classification result and the target feature map to obtain a corresponding target segmentation map; and a target risk prediction model is used to obtain a risk prediction result and an analysis result based on the target feature vector, the classification result, the target segmentation map and historical data of the to-be-detected gastroscope image. Thus, the early gastric cancer and precancerous lesion analysis method provided by the present application can accurately identify the type and lesion progression risk of precancerous lesions, thereby assisting doctors in accurately judging the disease condition and effectively preventing the occurrence of gastric cancer.

[0059] Additional aspects and advantages of the present application will be made apparent by the following description and the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0060] The above and / or additional aspects and advantages of the present application will become apparent and be more readily understood through consideration of the following description, taken in conjunction with the accompanying drawings, in which:

[0061] Figure 1 is a flowchart of an early gastric cancer and precancerous lesion analysis method according to an embodiment of the present application;

[0062] Figure 2 is a structural diagram of an early gastric cancer and precancerous lesion analysis device according to an embodiment of the present application;

[0063] Figure 3 is a block diagram of an electronic device suitable for an early gastric cancer and precancerous lesion analysis method according to an embodiment of the present application. DETAILED DESCRIPTION

[0064] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The technical solutions in the embodiments of the present application will be described in detail below with reference to the drawings and in combination with the embodiments.

[0065] In order for those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in combination with the drawings of the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should belong to the scope of protection of the present application.

[0066] The early gastric cancer and precancerous lesion analysis method and device according to the embodiments of the present application will be described below with reference to the drawings.

[0067] Figure 1 is a flowchart of the early gastric cancer and precancerous lesion analysis method of the embodiments of the present application, which is applied to a dental cleaning device.

[0068] As shown in Figure 1 , the method comprises:

[0069] S1, obtaining a gastroscopy image to be detected.

[0070] It should be noted that the execution subject of the above early gastric cancer and precancerous lesion analysis method is an early gastric cancer and precancerous lesion analysis device, which can be realized by software and / or hardware. The early gastric cancer and precancerous lesion analysis device in the embodiments can be configured in an electronic device.

[0071] In the present example embodiment, the electronic device can include terminal devices and servers, and the like. The embodiments of the present application are not limited to electronic devices.

[0072] In an embodiment of the present application, the above-mentioned gastroscopy image to be detected can be a raw image generated by a gastroscope.

[0073] S2, extracting features of the gastroscopy image to be detected by a target feature extraction deep network to obtain corresponding target feature vectors and target feature maps.

[0074] In an embodiment of the present application, after obtaining the gastroscopy image to be detected by the above-mentioned steps, the features of the gastroscopy image to be detected can be extracted by a target feature extraction deep network to obtain corresponding target feature vectors and target feature maps. In an embodiment of the present application, the above-mentioned target feature extraction deep network comprises a first input layer, a convolutional network layer, a noise reduction layer and a first output layer.

[0075] Specifically, in one embodiment of the present application, the method for extracting features of the gastroscopic image to be detected by the target feature extraction deep network to obtain the corresponding target feature vector and target feature map can include the following steps:

[0076] S21, the gastroscopic image to be detected is transmitted to the convolution network layer through the first input layer;

[0077] S22, the convolution network layer extracts features of the gastroscopic image to be detected through a plurality of convolution layers and SE modules to obtain a first feature image;

[0078] S23, the denoising layer removes noise of the first feature image to obtain a second feature image;

[0079] S24, the first output layer obtains the corresponding target feature vector and target feature map based on the second feature image.

[0080] In one embodiment of the present application, the convolution network layer includes a plurality of convolution layers, and each convolution layer has a corresponding SE module. In one embodiment of the present application, each convolution layer is composed of a plurality of convolution kernels (filters) to extract low-level, middle-level and high-level features in the gastroscopic image to be detected. The initial convolution layer extracts low-level features of edges and textures, the middle convolution layer extracts middle-level features of shapes and structures, and the high convolution layer extracts complex lesion features.

[0081] Specifically, in one embodiment of the present application, the method for the convolution network layer to extract features of the gastroscopic image to be detected through a plurality of convolution layers and SE modules to obtain a first feature image can include the following steps:

[0082] Step 1, extracting features of the gastroscopic image to be detected through the first convolution layer to obtain a third feature image;

[0083] Step 2, processing features of the third feature image through the first SE module to obtain a fourth feature image;

[0084] Step 3, extracting features of the fourth feature image through the second convolution layer to obtain a fifth feature image;

[0085] Step 4, processing features of the fifth feature image through the second SE module to obtain a sixth feature image;

[0086] Step 5, repeating the above steps, and determining the feature image output by the SE module corresponding to the last convolution layer as the first feature image.

[0087] In one embodiment of the present application, the first SE module includes a global average pooling layer, a first fully connected layer, a ReLU activation function layer, a second fully connected layer, a Sigmoid activation function layer, and an adaptive weighting layer.

[0088] In one embodiment of the present application, the method of performing feature processing on the third feature image through the first SE module to obtain a fourth feature image can include the following steps:

[0089] Step a, the global average pooling layer performs global average pooling on the third feature image to obtain a first feature vector;

[0090] Step b, the first fully connected layer performs dimension compression on the first feature vector to obtain a second feature vector;

[0091] Step c, the ReLU activation function layer applies the ReLU activation function to the second feature vector to obtain a third feature vector;

[0092] Step d, the second fully connected layer performs dimension expansion on the third feature vector to obtain a first weight coefficient of each channel;

[0093] Step e, the Sigmoid activation function layer performs Sigmoid activation on the first weight coefficient to obtain a second weight coefficient of each channel;

[0094] Step f, the adaptive weighting layer multiplies the second weight coefficient with the third feature image by channel to obtain the fourth feature image.

[0095] In one embodiment of the present application, the global average pooling layer performs global average pooling on each channel of the third feature image, compresses the third feature image into the average value of each channel, and generates the first feature vector, wherein the dimension of the first feature vector is 1x1xC.

[0096] In one embodiment of the present application, the first fully connected layer can compress the first feature vector to C / r dimensions, wherein r is a scaling coefficient which can be set as needed, for example, taking values of 4 or 8.

[0097] Further, in one embodiment of the present application, the second fully connected layer can expand the third feature vector to C dimensions to obtain the first weight coefficient of each channel.

[0098] In one embodiment of the present application, the Sigmoid activation function layer performs Sigmoid activation on the first weight coefficient to ensure that the second weight coefficient of each channel is between 0 and 1.

[0099] Further, in an embodiment of the present application, the noise reduction layer can use a sparse representation decomposition method and a compressed sensing technology to remove noise from the first feature image, convert the image features of the first feature image into a sparse basis representation, thereby effectively filtering out noise and further improving the accuracy of feature extraction of the second feature image.

[0100] In an embodiment of the present application, the target feature vector and the target feature map obtained by the above steps include rich information of the lesion area in the endoscopic image to be detected.

[0101] S3, type diagnosis of the target feature vector is performed by a target diagnosis model to obtain a corresponding classification result.

[0102] In an embodiment of the present application, after obtaining the target feature vector by the above steps, type diagnosis of the target feature vector can be performed by a target diagnosis model to obtain a corresponding classification result. In an embodiment of the present application, the target diagnosis model can include a second input layer, a third fully connected layer, a Dropout layer, a classification layer, and a second output layer.

[0103] Specifically, in an embodiment of the present application, the method of performing type diagnosis of the target feature vector by the target diagnosis model to obtain a corresponding classification result can include the following steps:

[0104] S31, the target feature vector is transmitted to the third fully connected layer through the second input layer;

[0105] S32, the third fully connected layer performs feature processing on the target feature vector to obtain a fourth feature vector;

[0106] S33, the fourth feature vector passes through the Dropout layer to obtain a fifth feature vector;

[0107] S34, the classification layer classifies the fifth feature vector by a Softmax function to obtain a probability vector;

[0108] S35, the second output layer model determines the class corresponding to the maximum probability value in the probability vector as the classification result and outputs it.

[0109] In an embodiment of the present application, the third fully connected layer can include a plurality of fully connected layers (Fully Connected Layers) for further processing of input features and extracting more discriminative features, and each fully connected layer has a corresponding ReLU activation function to introduce nonlinearity and enhance the learning ability of the target diagnosis model.

[0110] In an embodiment of the present application, the Dropout layer can randomly discard a portion of neurons during the training process to prevent overfitting and improve the generalization ability of the target diagnostic model.

[0111] Further, in an embodiment of the present application, the classification layer classifies the fifth feature vector through a Softmax function to obtain a probability vector, wherein the probability vector has a dimension of 4. In an embodiment of the present application, each dimension in the probability vector represents a class, and each class represents a pathological type, wherein the pathological types represented by the probability vector can include non-tumorous lesions (gastritis and intestinal epithelial metaplasia), low-grade intraepithelial neoplasia, high-grade intraepithelial neoplasia, and early gastric cancer.

[0112] S4, based on the classification result and the target feature map, marking the boundary and the lesion degree of the lesion area through a target boundary recognition model to obtain a corresponding target segmentation map.

[0113] In an embodiment of the present application, after obtaining the classification result and the target feature map through the above steps, the boundary and the lesion degree of the lesion area can be marked through a target boundary recognition model based on the classification result and the target feature map to obtain a corresponding target segmentation map. In an embodiment of the present application, the target boundary recognition model includes a third input layer, an encoder, a skip connection layer, a decoder, a multi-scale feature fusion layer, and a third output layer.

[0114] Specifically, in an embodiment of the present application, the method of marking the boundary and the lesion degree of the lesion area through a target boundary recognition model based on the classification result and the target feature map to obtain a corresponding target segmentation map can include the following steps:

[0115] S41, the classification result and the target feature map are transmitted to the encoder through the third input layer;

[0116] S42, the encoder performs down-sampling on the target feature map to extract spatial features of different scales to obtain a seventh feature image;

[0117] S43, the skip connection layer transmits the features of each layer of the encoder to the corresponding layer of the decoder;

[0118] S44, the decoder restores the spatial resolution of the seventh feature image layer by layer to obtain a first segmentation map corresponding to the target feature map;

[0119] S45, the multi-scale feature fusion layer fuses the first segmentation map to obtain a second segmentation map;

[0120] S46, the third output layer obtains a corresponding target segmentation map based on the classification result and the second segmentation map.

[0121] In an embodiment of the present application, the above-mentioned encoder can be composed of multi-layer convolutional layers and pooling layers, which down-sample the input target feature map layer by layer to extract spatial features of different scales and obtain the seventh feature image.

[0122] In an embodiment of the present application, a skip connection layer is introduced between each layer of the above-mentioned encoder and the corresponding layer of the decoder to ensure that important spatial information is not lost during the decoding process of the subsequent decoder.

[0123] Further, in an embodiment of the present application, the above-mentioned decoder can be composed of convolutional layers and up-sampling layers to recover the spatial resolution of the seventh feature image layer by layer and obtain the first segmentation map with the same size as the target feature map. After the convolution operation of each layer, an SE module is introduced to weight each layer of features in the channel to improve the boundary recognition ability of the target boundary recognition model in complex background.

[0124] In an embodiment of the present application, the above-mentioned multi-scale feature fusion module can fuse feature maps of different levels to accurately identify the boundaries of lesions of different scales and capture more subtle boundary information.

[0125] In an embodiment of the present application, the above-mentioned third output layer outputs a segmentation map corresponding to the classification result of each channel, where each channel corresponds to a type (such as chronic atrophic gastritis, intestinal metaplasia, low-grade intraepithelial neoplasia, and high-grade intraepithelial neoplasia), and the target segmentation map can mark the boundaries and lesion degrees of each lesion region.

[0126] S5, based on the target feature vector, the classification result, the target segmentation map, and the historical data of the gastroscope image to be detected, obtaining a risk prediction result and an analysis result through a target risk prediction model.

[0127] In an embodiment of the present application, by obtaining the target feature vector, the classification result, and the target segmentation map through the above-mentioned steps, a risk prediction result and an analysis result can be obtained through a target risk prediction model based on the target feature vector, the classification result, the target segmentation map, and the historical data of the gastroscope image to be detected. In an embodiment of the present application, the above-mentioned target risk prediction model can include a fourth input layer, a time series analysis layer, a risk assessment layer, and a fourth output layer.

[0128] Specifically, in an embodiment of the present application, the method of obtaining a risk prediction result and an analysis result through a target risk prediction model based on the target feature vector, the classification result, the target segmentation map, and the historical data of the gastroscope image to be detected can include the following steps:

[0129] S51, obtain historical data of the gastroscope image to be detected;

[0130] S52, the target feature vector, the classification result, the target segmentation map and the historical data of the gastroscope image to be detected are transmitted to the time sequence analysis layer through the fourth input layer;

[0131] S53, the time sequence analysis layer obtains the corresponding sixth feature vector by using the LSTM network through the target feature vector, the classification result, the target segmentation map and the historical data of the gastroscope image to be detected;

[0132] S54, the risk assessment layer analyzes the sixth feature vector to obtain a predicted risk value;

[0133] S55, the fourth output layer outputs the predicted risk value as a risk prediction result;

[0134] S56, based on the risk prediction result, a corresponding analysis result is obtained.

[0135] In an embodiment of the present application, the above-mentioned gastroscope image to be detected can be the clinical medical record, pathological report and gastroscope image of the previous gastroscope examination of the patient.

[0136] In an embodiment of the present application, the time sequence analysis layer can use the LSTM network (Long Short Term Memory network) to process time-dependent information to capture the rules of the evolution of the lesion over time, and predict the future development trend of the lesion through multiple time step feature inputs.

[0137] Further, in an embodiment of the present application, the risk assessment layer can calculate the risk of the patient developing into high-grade intraepithelial neoplasia or gastric cancer in a future preset time through a fully connected layer and a logistic regression layer. The preset time can be set as needed, for example, 6 months or 1 year.

[0138] In an embodiment of the present application, the risk prediction result can be a risk score to represent the probability value of the patient developing into a high-grade lesion in the future, wherein different risk scores correspond to different probability values. The corresponding relationship between the risk score and the probability value can be saved in advance.

[0139] Further, in an embodiment of the present application, after obtaining the risk prediction result through the above steps, the doctor can use the risk prediction result as the basis for clinical decision-making to formulate individualized treatment and follow-up strategies, and output the treatment and follow-up strategies as the analysis result.

[0140] In an embodiment of the present application, before the risk prediction result and the analysis result corresponding to the gastroscopic image to be detected are obtained by the above method, the above method can further include: obtaining a gastroscopic image training dataset, and training the initial feature extraction deep network, the initial diagnosis model, the initial boundary recognition model, and the initial risk prediction model using the gastroscopic image training dataset to obtain the target feature extraction deep network, the target diagnosis model, the target boundary recognition model, and the target risk prediction model.

[0141] In an embodiment of the present application, a gastroscopic image dataset of early gastric cancer and precancerous lesions can be obtained, and the gastroscopic image data in the gastroscopic image dataset is labeled by senior physicians with biopsy or surgical pathology reports as the gold standard, and the labeled image dataset is determined as the gastroscopic image training dataset.

[0142] In an embodiment of the present application, the method of training the initial feature extraction deep network, the initial diagnosis model, the initial boundary recognition model, and the initial risk prediction model using the gastroscopic image training dataset is the same as the prior art, and the present disclosure embodiment will not be described here.

[0143] The embodiment of the present application proposes an analysis method for early gastric cancer and precancerous lesions, obtains a gastroscopic image to be detected, extracts features of the gastroscopic image to be detected through a target feature extraction deep network to obtain a corresponding target feature vector and a target feature map, performs type diagnosis on the target feature vector through a target diagnosis model to obtain a corresponding classification result, marks the boundary of the lesion area and the degree of lesion based on the classification result and the target feature map through a target boundary recognition model to obtain a corresponding target segmentation map, and obtains a risk prediction result and an analysis result based on the target feature vector, the classification result, the target segmentation map, and historical data of the gastroscopic image to be detected through a target risk prediction model. Therefore, the analysis method for early gastric cancer and precancerous lesions proposed by the present application can accurately identify the type and progression risk of precancerous lesions, thereby assisting physicians in accurately judging the disease condition and avoiding underestimating the disease condition, and effectively preventing the occurrence of gastric cancer.

[0144] To achieve the above-mentioned embodiments, as Figure 2 shown, the present embodiment further provides an analysis device 10 for early gastric cancer and precancerous lesions, which comprises an acquisition module 100, an extraction module 200, a classification module 300, a processing module 400, and a risk prediction module 500.

[0145] The acquisition module 100 is configured to acquire a gastroscopic image to be detected.

[0146] The extraction module 200 is configured to extract features of the gastroscopic image to be detected through a target feature extraction deep network to obtain a corresponding target feature vector and a target feature map.

[0147] The classification module 300 is configured to perform type diagnosis on the target feature vector by using a target diagnosis model, and obtain a corresponding classification result.

[0148] The processing module 400 is configured to mark the boundary and the lesion degree of the lesion area by using a target boundary recognition model based on the classification result and the target feature map, and obtain a corresponding target segmentation map.

[0149] The risk prediction module 500 is configured to obtain a risk prediction result and an analysis result by using a target risk prediction model based on the target feature vector, the classification result, the target segmentation map and historical data of the to-be-detected gastroscope image.

[0150] In an embodiment of the present application, the target feature extraction deep network comprises a first input layer, a convolutional network layer, a noise reduction layer and a first output layer.

[0151] The to-be-detected gastroscope image is transmitted to the convolutional network layer through the first input layer.

[0152] The convolutional network layer extracts features of the to-be-detected gastroscope image by using a plurality of convolutional layers and SE modules, and obtains a first feature image.

[0153] The noise reduction layer removes noise from the first feature image, and obtains a second feature image.

[0154] The first output layer obtains a corresponding target feature vector and target feature map based on the second feature image.

[0155] In an embodiment of the present application, the extraction module is further configured to:

[0156] The first convolutional layer extracts features of the to-be-detected gastroscope image, and obtains a third feature image.

[0157] The first SE module processes features of the third feature image, and obtains a fourth feature image.

[0158] The second convolutional layer extracts features of the fourth feature image, and obtains a fifth feature image.

[0159] The second SE module processes features of the fifth feature image, and obtains a sixth feature image.

[0160] The above steps are repeated, and the feature image output by the SE module corresponding to the last convolutional layer is determined as the first feature image.

[0161] In an embodiment of the present application, the first SE module includes a global average pooling layer, a first full connection layer, a ReLU activation function layer, a second full connection layer, a Sigmoid activation function layer, and an adaptive weighting layer; the extraction module is further configured to:

[0162] The global average pooling layer performs global average pooling on the third feature image to obtain a first feature vector;

[0163] The first full connection layer performs dimension compression on the first feature vector to obtain a second feature vector;

[0164] The ReLU activation function layer applies a ReLU activation function to the second feature vector to obtain a third feature vector;

[0165] The second full connection layer performs dimension expansion on the third feature vector to obtain a first weight coefficient of each channel;

[0166] The Sigmoid activation function layer performs Sigmoid activation on the first weight coefficient to obtain a second weight coefficient of each channel;

[0167] The adaptive weighting layer multiplies the second weight coefficient and the third feature image by channel to obtain a fourth feature image.

[0168] In an embodiment of the present application, the target diagnosis model includes a second input layer, a third full connection layer, a Dropout layer, a classification layer, and a second output layer; the classification module is specifically configured to:

[0169] The target feature vector is transmitted to the third full connection layer through the second input layer;

[0170] The third full connection layer performs feature processing on the target feature vector to obtain a fourth feature vector;

[0171] The fourth feature vector passes through the Dropout layer to obtain a fifth feature vector;

[0172] The classification layer classifies the fifth feature vector through a Softmax function to obtain a probability vector;

[0173] The second output layer model determines the class corresponding to the maximum probability value in the probability vector as a classification result and outputs the classification result.

[0174] In an embodiment of the present application, the target boundary recognition model includes a third input layer, an encoder, a skip connection layer, a decoder, a multi-scale feature fusion layer, and a third output layer; the processing module is specifically configured to:

[0175] The classification result and the target feature image are transmitted to the encoder through the third input layer;

[0176] The encoder down-samples the target feature map to extract spatial features of different scales to obtain a seventh feature image;

[0177] The skip connection layer transmits each layer feature of the encoder to the corresponding layer of the decoder;

[0178] The decoder recovers the spatial resolution of the seventh feature image layer by layer to obtain the first segmentation map corresponding to the target feature map;

[0179] The multi-scale feature fusion layer fuses the first segmentation map to obtain a second segmentation map;

[0180] The third output layer obtains the corresponding target segmentation map based on the classification result and the second segmentation map.

[0181] In an embodiment of the present application, the above-mentioned target risk prediction model comprises a fourth input layer, a time series analysis layer, a risk assessment layer and a fourth output layer; the risk prediction result and the analysis result are obtained by the target risk prediction model based on the target feature vector, the classification result, the target segmentation map and the historical data of the to-be-detected gastroscope image, comprising:

[0182] The historical data of the to-be-detected gastroscope image is obtained;

[0183] The target feature vector, the classification result, the target segmentation map and the historical data of the to-be-detected gastroscope image are transmitted to the time series analysis layer through the fourth input layer;

[0184] The time series analysis layer obtains the corresponding sixth feature vector by using the LSTM network based on the target feature vector, the classification result, the target segmentation map and the historical data of the to-be-detected gastroscope image;

[0185] The risk assessment layer analyzes the sixth feature vector to obtain a predicted risk value;

[0186] The fourth output layer outputs the predicted risk value as the risk prediction result;

[0187] Based on the risk prediction result, the corresponding analysis result is obtained.

[0188] According to the early gastric cancer and precancerous lesion analysis device provided by the embodiment of the application, a gastroscope image to be detected is acquired; a feature of the gastroscope image to be detected is extracted through a target feature extraction deep network, and a corresponding target feature vector and a target feature map are obtained; a type diagnosis is performed on the target feature vector through a target diagnosis model, and a corresponding classification result is obtained; based on the classification result and the target feature map, a boundary of a lesion area and a lesion degree are marked through a target boundary recognition model, and a corresponding target segmentation map is obtained; and based on the target feature vector, the classification result, the target segmentation map and historical data of the gastroscope image to be detected, a risk prediction result and an analysis result are obtained through a target risk prediction model. Therefore, the early gastric cancer and precancerous lesion analysis method provided by the application can accurately identify the type and lesion progression risk of precancerous lesions, so as to assist doctors in accurately judging the disease condition and effectively preventing the occurrence of gastric cancer.

[0189] As shown in Figure 3 The electronic device 300 includes a computing unit 301 that can perform various appropriate actions and processes in accordance with a computer program stored in a read-only memory (ROM) 302 or a computer program loaded into a random access memory (RAM) 303 from a storage unit 308. Various programs and data required for the operation of the electronic device 300 can also be stored in the RAM 303. The computing unit 301, the ROM 302, and the RAM 303 are connected to each other through a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0190] Various components in the electronic device 300 are connected to the I / O interface 305, including an input unit 308 such as a keyboard, a mouse, and the like; an output unit 307 such as various types of displays, a speaker, and the like; a storage unit 308 such as a magnetic disk, an optical disk, and the like; and a communication unit 309 such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 309 allows the electronic device 300 to exchange information / data with other electronic devices through a computer network such as the Internet and / or various telecommunication networks.

[0191] The computing unit 301 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 301 performs various methods and processes described above, such as the analysis method of early gastric cancer and precancerous lesions. For example, in some embodiments, the analysis method of early gastric cancer and precancerous lesions can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 308. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 300 via the ROM 302 and / or the communication unit 309. When the computer program is loaded into the RAM 303 and executed by the computing unit 301, one or more steps of the analysis method of early gastric cancer and precancerous lesions described above can be performed. Alternatively, in other embodiments, the computing unit 301 can be configured to perform the analysis method of early gastric cancer and precancerous lesions by any other appropriate means, such as by means of firmware.

[0192] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic electronic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input system, and at least one output system.

[0193] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing system to produce a machine, such that the program code, when executed by the processor or controller, implements the functions / acts specified in the flowcharts and / or block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0194] In the context of this disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, system, or electronic device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or electronic device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include a tangible, non-transitory memory such as a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0195] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display system (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing system (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of systems can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0196] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0197] The computer system can include clients and servers. This relationship can be. The servers are generally remote from the users and can be accessed via the Internet using a communication network. The relationship can be a client-server relationship over a communications network, and as such, the servers can be accessed by the clients using computer programs. The servers can be cloud servers, also known as cloud computing servers or cloud hosts, which are mainframe products in the cloud computing service system, and solve the defects of large management difficulty and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or simply "VPS").

[0198] It should be understood that the various forms of flow shown above can be reordered, added to, or have steps deleted. For example, the steps described in the present disclosure can be performed in parallel, in series, or in a different order, as long as the desired results of the technical solutions of the present disclosure can be achieved, and the present disclosure is not limited herein.

[0199] The above detailed description does not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. An analysis method of early gastric cancer and precancerous lesions, characterized by, The method comprises: obtaining a gastroscopy image to be detected; extracting features of the gastroscopy image to be detected through a target feature extraction deep network to obtain a corresponding target feature vector and a target feature map; the target feature vector is transmitted to a third fully connected layer of the target diagnosis model through a second input layer of the target diagnosis model, the third fully connected layer performs feature processing on the target feature vector to obtain a fourth feature vector, the fourth feature vector is obtained through a Dropout layer of the target diagnosis model to obtain a fifth feature vector, a classification layer of the target diagnosis model classifies the fifth feature vector through a Softmax function to obtain a probability vector, and a second output layer of the target diagnosis model determines a class corresponding to a maximum probability value in the probability vector as a classification result corresponding to the target feature vector and outputs the classification result; based on the classification result and the target feature map, a target boundary recognition model is used to mark the boundary and the lesion degree of the lesion area to obtain a corresponding target segmentation map; based on the target feature vector, the classification result, the target segmentation map and historical data of the gastroscopy image to be detected, a target risk prediction model is used to obtain a risk prediction result and an analysis result.

2. The method of claim 1, wherein, The target feature extraction deep network comprises a first input layer, a convolutional network layer, a noise reduction layer and a first output layer; the target feature extraction deep network extracts features of the gastroscopy image to be detected to obtain a corresponding target feature vector and a target feature map, which comprises: the gastroscopy image to be detected is transmitted to the convolutional network layer through the first input layer; the convolutional network layer extracts features of the gastroscopy image to be detected through a plurality of convolutional layers and SE modules to obtain a first feature image; the noise reduction layer removes noise from the first feature image to obtain a second feature image; the first output layer obtains a corresponding target feature vector and a target feature map based on the second feature image.

3. The method of claim 2, wherein, Each convolutional layer has a corresponding SE module; the convolutional network layer extracts features of the gastroscopy image to be detected through a plurality of convolutional layers and SE modules to obtain a first feature image, which comprises: a first convolutional layer is used to extract features of the gastroscopy image to be detected to obtain a third feature image; a first SE module is used to process features of the third feature image to obtain a fourth feature image; a second convolutional layer is used to extract features of the fourth feature image to obtain a fifth feature image; a second SE module is used to process features of the fifth feature image to obtain a sixth feature image; the above steps are repeated, and a feature image output by a SE module corresponding to a last convolutional layer is determined as the first feature image.

4. The method of claim 3, wherein, The first SE module comprises a global average pooling layer, a first fully connected layer, a ReLU activation function layer, a second fully connected layer, a Sigmoid activation function layer and an adaptive weighting layer; the first SE module is used to process features of the third feature image to obtain a fourth feature image, which comprises: the global average pooling layer performs global average pooling on the third feature image to obtain a first feature vector; The first fully connected layer performs dimension compression on the first feature vector to obtain a second feature vector; The ReLU activation function layer applies a ReLU activation function to the second feature vector to obtain a third feature vector; The second fully connected layer performs dimension expansion on the third feature vector to obtain a first weight coefficient of each channel; The Sigmoid activation function layer performs Sigmoid activation on the first weight coefficient to obtain a second weight coefficient of each channel; The adaptive weighting layer multiplies the second weight coefficient and the third feature image by channel to obtain a fourth feature image.

5. The method of claim 1, wherein, The target boundary recognition model comprises a third input layer, an encoder, a skip connection layer, a decoder, a multi-scale feature fusion layer and a third output layer; based on the classification result and the target feature map, the boundary and the lesion degree of the lesion area are marked by the target boundary recognition model to obtain the corresponding target segmentation map, comprising: The classification result and the target feature map are transmitted to the encoder through the third input layer; The encoder extracts spatial features of different scales by downsampling the target feature map to obtain a seventh feature image; The skip connection layer transmits the features of each layer of the encoder to the corresponding layer of the decoder; The decoder recovers the spatial resolution of the seventh feature image layer by layer to obtain a first segmentation map corresponding to the target feature map; The multi-scale feature fusion layer fuses the first segmentation map to obtain a second segmentation map; The third output layer obtains the corresponding target segmentation map based on the classification result and the second segmentation map.

6. The method of claim 1, wherein, The target risk prediction model comprises a fourth input layer, a time series analysis layer, a risk assessment layer and a fourth output layer; based on the target feature vector, the classification result, the target segmentation map and the historical data of the to-be-detected gastroscope image, the risk prediction result and the analysis result are obtained by the target risk prediction model, comprising: Obtain the historical data of the to-be-detected gastroscope image; The target feature vector, the classification result, the target segmentation map and the historical data of the to-be-detected gastroscope image are transmitted to the time series analysis layer through the fourth input layer; The time series analysis layer obtains a corresponding sixth feature vector by using the LSTM network through the target feature vector, the classification result, the target segmentation map and the historical data of the to-be-detected gastroscope image; The risk assessment layer analyzes the sixth feature vector to obtain a predicted risk value; The fourth output layer outputs the predicted risk value as a risk prediction result; Based on the risk prediction result, the corresponding analysis result is obtained.

7. An apparatus for analyzing early gastric cancer and precancerous lesions, characterized by comprising: The device comprises: An acquisition module for acquiring a to-be-detected gastroscope image; An extraction module for extracting features of the to-be-detected gastroscope image by a target feature extraction deep network to obtain a corresponding target feature vector and a target feature map; The classification module is configured to transmit the target feature vector through a second input layer of the target diagnosis model to a third fully connected layer of the target diagnosis model, perform feature processing on the target feature vector by the third fully connected layer to obtain a fourth feature vector, transmit the fourth feature vector through a Dropout layer of the target diagnosis model to obtain a fifth feature vector, perform classification on the fifth feature vector by a classification layer of the target diagnosis model through a Softmax function to obtain a probability vector, and determine a class corresponding to a maximum probability value in the probability vector as a classification result corresponding to the target feature vector and output the classification result. The processing module is configured to mark a boundary and a lesion degree of a lesion region by a target boundary recognition model based on the classification result and the target feature map to obtain a corresponding target segmentation map. The risk prediction module is configured to obtain a risk prediction result and an analysis result by a target risk prediction model based on the target feature vector, the classification result, the target segmentation map and historical data of the to-be-detected gastroscope image. 8.An electronic device, comprising: at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.

9. A computer storage medium, wherein, The computer storage medium stores computer executable instructions; the computer executable instructions are executed by the processor to realize the method in any one of claims 1-6.

Images