Analysis method and device for early gastric cancer and precancerous lesion and storage medium

Abstract

The invention discloses an early gastric cancer and precancerous lesion analysis method and device and a storage medium. The method comprises the following steps: acquiring a to-be-detected gastroscope image; extracting features of the to-be-detected gastroscope image through a target feature extraction deep network to obtain a corresponding target feature vector and a target feature map; performing type diagnosis on the target feature vector through a target diagnosis model to obtain a corresponding classification result; based on the classification result and the target feature map, marking the boundary and the lesion degree of the lesion area through a target boundary recognition model to obtain a corresponding target segmentation map; and based on the target feature vector, the classification result, the target segmentation image and historical data of the to-be-detected gastroscope image, obtaining a risk prediction result and an analysis result through a target risk prediction model. Therefore, the precancerous lesion type and lesion progress risk are accurately identified, so that a doctor is assisted to accurately judge the illness state, wrong underestimation of the illness state is avoided, and gastric cancer is effectively prevented.

Description

Technical Field

[0001] The present invention relates to the field of clean artificial intelligence technology, and in particular to an analysis method, device and storage medium for early gastric cancer and precancerous lesions. Background Art

[0002] At present, gastric cancer is one of the most common cancers in the world. The development of gastric cancer is a multi-stage process, which may include different stages of chronic atrophic gastritis, intestinal metaplasia, low-grade intraepithelial neoplasia, and high-grade intraepithelial neoplasia. Among them, the occurrence process of gastric cancer can be: normal gastric mucosa → chronic inflammation → chronic atrophic gastritis → intestinal metaplasia → low-grade intraepithelial neoplasia → high-grade intraepithelial neoplasia → early gastric cancer.

[0003] In the existing technology, artificial intelligence can be used to assist digestive endoscopy in identifying gastric cancer and determining the depth of gastric cancer infiltration, but it is unable to identify or accurately identify the various stages of precancerous lesions, which may lead to the inability to effectively prevent the occurrence of gastric cancer. Summary of the invention

[0004] The present invention aims to solve one of the technical problems in the related art at least to a certain extent.

[0005] To this end, the present invention proposes an analysis method for early gastric cancer and precancerous lesions, which can accurately identify the type of precancerous lesions and the risk of lesion progression, thereby assisting physicians in making accurate judgments on the condition, avoiding erroneous underestimation of the condition, and effectively preventing the occurrence of gastric cancer.

[0006] Another object of the present invention is to provide an analysis device for early gastric cancer and precancerous lesions.

[0007] To achieve the above object, the present invention provides a method for analyzing early gastric cancer and precancerous lesions, the method comprising:

[0008] Acquire a gastroscopic image to be detected;

[0009] Extracting the features of the gastroscope image to be detected through a target feature extraction deep network to obtain a corresponding target feature vector and a target feature map;

[0010] Performing type diagnosis on the target feature vector through a target diagnosis model to obtain a corresponding classification result;

[0011] Based on the classification result and the target feature map, the boundary and the degree of the lesion area are marked by a target boundary recognition model to obtain a corresponding target segmentation map;

[0012] Based on the target feature vector, the classification result, the target segmentation map and the historical data of the gastroscopic image to be detected, a risk prediction result and an analysis result are obtained through a target risk prediction model.

[0013] The method for analyzing early gastric cancer and precancerous lesions according to the embodiment of the present invention may also have the following additional technical features:

[0014] In one embodiment of the present invention, the target feature extraction deep network includes a first input layer, a convolutional network layer, a noise reduction layer and a first output layer; the feature extraction of the gastroscope image to be detected by the target feature extraction deep network to obtain the corresponding target feature vector and target feature map includes:

[0015] The gastroscope image to be detected is transmitted to the convolutional network layer through the first input layer;

[0016] The convolutional network layer extracts features of the gastroscope image to be detected through multiple convolutional layers and SE modules to obtain a first feature image;

[0017] The denoising layer performs noise removal processing on the first feature image to obtain a second feature image;

[0018] The first output layer obtains a corresponding target feature vector and a target feature map based on the second feature image.

[0019] In one embodiment of the present invention, each convolutional layer is followed by a corresponding SE module; the convolutional network layer extracts features of the gastroscopic image to be detected through multiple convolutional layers and SE modules to obtain a first feature image, including:

[0020] Extracting features of the gastroscope image to be detected through the first convolutional layer to obtain a third feature image;

[0021] Performing feature processing on the third feature image by the first SE module to obtain a fourth feature image;

[0022] Performing feature extraction on the fourth feature image through a second convolutional layer to obtain a fifth feature image;

[0023] Performing feature processing on the fifth feature image by a second SE module to obtain a sixth feature image;

[0024] Repeat the above steps, and determine the feature image output by the last convolutional layer corresponding to the SE module as the first feature image.

[0025] In one embodiment of the present invention, the first SE module includes a global average pooling layer, a first fully connected layer, a ReLU activation function layer, a second fully connected layer, a Sigmoid activation function layer and an adaptive weighting layer; the third feature image is subjected to feature processing by the first SE module to obtain a fourth feature image, including:

[0026] The global average pooling layer performs global average pooling on the third feature image to obtain a first feature vector;

[0027] The first fully connected layer performs dimension compression on the first feature vector to obtain a second feature vector;

[0028] The ReLU activation function layer applies a ReLU activation function to the second eigenvector to obtain a third eigenvector;

[0029] The second fully connected layer performs dimension expansion on the third feature vector to obtain a first weight coefficient for each channel;

[0030] The Sigmoid activation function layer performs Sigmoid activation on the first weight coefficient to obtain a second weight coefficient for each channel;

[0031] The adaptive weighting layer multiplies the second weight coefficient by the third feature image by channel to obtain a fourth feature image.

[0032] In one embodiment of the present invention, the target diagnosis model includes a second input layer, a third fully connected layer, a Dropout layer, a classification layer, and a second output layer; the target diagnosis model is used to perform type diagnosis on the target feature vector to obtain a corresponding classification result, including:

[0033] The target feature vector is transmitted to the third fully connected layer through the second input layer;

[0034] The third fully connected layer performs feature processing on the target feature vector to obtain a fourth feature vector;

[0035] The fourth eigenvector passes through the Dropout layer to obtain a fifth eigenvector;

[0036] The classification layer classifies the fifth feature vector by using a Softmax function to obtain a probability vector;

[0037] The second output layer determines the category corresponding to the maximum probability value in the probability vector as the classification result and outputs it.

[0038] In one embodiment of the present invention, the target boundary recognition model includes a third input layer, an encoder, a skip connection layer, a decoder, a multi-scale feature fusion layer and a third output layer; based on the classification result and the target feature map, the boundary and the degree of the lesion area are marked by the target boundary recognition model to obtain the corresponding target segmentation map, including:

[0039] The classification result and the target feature map are transmitted to the encoder through the third input layer;

[0040] The encoder downsamples the target feature map to extract spatial features of different scales to obtain a seventh feature image;

[0041] The skip connection layer transmits each layer feature of the encoder to the corresponding layer of the decoder;

[0042] The decoder restores the spatial resolution of the seventh feature image layer by layer to obtain a first segmentation map corresponding to the target feature map;

[0043] The multi-scale feature fusion layer fuses the first segmentation map to obtain a second segmentation map;

[0044] The third output layer obtains a corresponding target segmentation map based on the classification result and the second segmentation map.

[0045] In one embodiment of the present invention, the target risk prediction model includes a fourth input layer, a time series analysis layer, a risk assessment layer and a fourth output layer; the risk prediction result and the analysis result are obtained by the target risk prediction model based on the target feature vector, the classification result, the target segmentation map and the historical data of the gastroscope image to be detected, including:

[0046] Acquiring historical data of the gastroscopic image to be detected;

[0047] The target feature vector, classification result, target segmentation map and historical data of the gastroscope image to be detected are transmitted to the time series analysis layer through the fourth input layer;

[0048] The time series analysis layer obtains the corresponding sixth eigenvector by using the LSTM network through the target eigenvector, the classification result, the target segmentation map and the historical data of the gastroscope image to be detected;

[0049] The risk assessment layer analyzes the sixth eigenvector to obtain a predicted risk value;

[0050] The fourth output layer outputs the predicted risk value as a risk prediction result;

[0051] Based on the risk prediction results, corresponding analysis results are obtained.

[0052] To achieve the above object, the present invention provides, on the other hand, an analysis device for early gastric cancer and precancerous lesions, the device comprising:

[0053] An acquisition module, used for acquiring a gastroscopic image to be detected;

[0054] An extraction module, used for extracting the features of the gastroscopic image to be detected through a target feature extraction deep network to obtain a corresponding target feature vector and a target feature map;

[0055] A classification module, used to perform type diagnosis on the target feature vector through a target diagnosis model to obtain a corresponding classification result;

[0056] A processing module, used for marking the boundary and degree of the lesion area through a target boundary recognition model based on the classification result and the target feature map, to obtain a corresponding target segmentation map;

[0057] The risk prediction module is used to obtain risk prediction results and analysis results through a target risk prediction model based on the target feature vector, the classification result, the target segmentation map and the historical data of the gastroscopic image to be detected.

[0058] The analysis method, device and storage medium of early gastric cancer and precancerous lesions of the embodiment of the present invention obtains a gastroscopic image to be detected; extracts the features of the gastroscopic image to be detected through a target feature extraction deep network to obtain a corresponding target feature vector and a target feature map; performs type diagnosis on the target feature vector through a target diagnosis model to obtain a corresponding classification result; based on the classification result and the target feature map, the boundary of the lesion area and the degree of lesion are marked through a target boundary recognition model to obtain a corresponding target segmentation map; based on the target feature vector, the classification result, the target segmentation map and the historical data of the gastroscopic image to be detected, the risk prediction result and the analysis result are obtained through a target risk prediction model. Therefore, the analysis method of early gastric cancer and precancerous lesions proposed by the present invention can accurately identify the type of precancerous lesions and the risk of lesion progression, thereby assisting doctors in making accurate judgments on the condition, avoiding erroneous underestimation of the condition, and effectively preventing the occurrence of gastric cancer.

[0059] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0061] Figure 1 is a flow chart of a method for analyzing early gastric cancer and precancerous lesions according to an embodiment of the present invention;

[0062] Figure 2 is a structural diagram of an analysis device for early gastric cancer and precancerous lesions according to an embodiment of the present invention;

[0063] Figure 3 The block diagram of an electronic device applicable to an analysis method for early gastric cancer and precancerous lesions according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0064] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0065] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0066] The following describes the method and device for analyzing early gastric cancer and precancerous lesions according to the embodiments of the present invention with reference to the accompanying drawings.

[0067] Figure 1 It is a flow chart of the analysis method of early gastric cancer and precancerous lesions according to an embodiment of the present invention, which is applied to a tooth cleaning device.

[0068] like Figure 1 As shown, the method includes:

[0069] S1, obtaining a gastroscopic image to be detected.

[0070] Among them, it should be noted that the executor of the above-mentioned early gastric cancer and precancerous lesions analysis method is the early gastric cancer and precancerous lesions analysis device, and the early gastric cancer and precancerous lesions analysis device can be implemented by software and / or hardware. The early gastric cancer and precancerous lesions analysis device in this embodiment can be configured in an electronic device.

[0071] In this example embodiment, the electronic device may include devices such as terminal devices and servers, and this embodiment does not limit the electronic device.

[0072] In one embodiment of the present invention, the above-mentioned gastroscope image to be detected may be an original image generated by a gastroscope.

[0073] S2, extracting the features of the gastroscope image to be detected through the target feature extraction deep network to obtain the corresponding target feature vector and target feature map.

[0074] In one embodiment of the present invention, after obtaining the gastroscopic image to be detected through the above steps, the features of the gastroscopic image to be detected can be extracted through a target feature extraction deep network to obtain a corresponding target feature vector and target feature map. In one embodiment of the present invention, the target feature extraction deep network includes a first input layer, a convolutional network layer, a noise reduction layer and a first output layer.

[0075] Specifically, in one embodiment of the present invention, the method of extracting the features of the gastroscope image to be detected by using a target feature extraction deep network to obtain the corresponding target feature vector and target feature map may include the following steps:

[0076] S21, the gastroscope image to be detected is transmitted to the convolutional network layer through the first input layer;

[0077] S22, the convolutional network layer extracts features of the gastroscope image to be detected through multiple convolutional layers and SE modules to obtain a first feature image;

[0078] S23, a denoising layer performs noise removal processing on the first feature image to obtain a second feature image;

[0079] S24, the first output layer obtains a corresponding target feature vector and a target feature map based on the second feature image.

[0080] In one embodiment of the present invention, the convolutional network layer includes multiple convolutional layers, and each convolutional layer is followed by a corresponding SE module. In one embodiment of the present invention, each convolutional layer is composed of a number of convolutional filters, which respectively extract low-level, intermediate and high-level features in the gastroscopic image to be detected, and the initial convolutional layer extracts low-level features of edges and textures, the intermediate convolutional layer extracts intermediate features of shapes and structures, and the high-level convolutional layer extracts complex lesion features.

[0081] Specifically, in one embodiment of the present invention, the convolutional network layer extracts features from the gastroscope image to be detected through multiple convolutional layers and SE modules, and the method for obtaining the first feature image may include the following steps:

[0082] Step 1: extract features of the gastroscope image to be detected through the first convolutional layer to obtain a third feature image;

[0083] Step 2: performing feature processing on the third feature image through the first SE module to obtain a fourth feature image;

[0084] Step 3: extract features from the fourth feature image through the second convolutional layer to obtain a fifth feature image;

[0085] Step 4: performing feature processing on the fifth feature image through the second SE module to obtain a sixth feature image;

[0086] Step 5: Repeat the above steps, and determine the feature image output by the last convolutional layer corresponding to the SE module as the first feature image.

[0087] Among them, in one embodiment of the present invention, the above-mentioned first SE module includes a global average pooling layer, a first fully connected layer, a ReLU activation function layer, a second fully connected layer, a Sigmoid activation function layer and an adaptive weighting layer.

[0088] Furthermore, in one embodiment of the present invention, the method of performing feature processing on the third feature image by the first SE module to obtain the fourth feature image may include the following steps:

[0089] Step a: a global average pooling layer performs global average pooling on the third feature image to obtain a first feature vector;

[0090] Step b: The first fully connected layer performs dimension compression on the first feature vector to obtain a second feature vector;

[0091] Step c: The ReLU activation function layer applies the ReLU activation function to the second eigenvector to obtain a third eigenvector;

[0092] Step d: The second fully connected layer performs dimension expansion on the third feature vector to obtain a first weight coefficient for each channel;

[0093] Step e: The Sigmoid activation function layer performs Sigmoid activation on the first weight coefficient to obtain a second weight coefficient for each channel;

[0094] Step f: The adaptive weighting layer multiplies the second weight coefficient by the third feature image by channel to obtain a fourth feature image.

[0095] In one embodiment of the present invention, the global average pooling layer performs global average pooling on each channel of the third feature image, compresses the third feature image into the average value of each channel, and generates a first feature vector, wherein the dimension of the first feature vector is 1×1×C.

[0096] And, in one embodiment of the present invention, the above-mentioned first fully connected layer can compress the first feature vector to C / r dimension, where r is a scaling factor and can be set as needed, for example, a value of 4 or 8.

[0097] Furthermore, in one embodiment of the present invention, the second fully connected layer may expand the third eigenvector back to C dimension to obtain a first weight coefficient for each channel.

[0098] And, in one embodiment of the present invention, the above-mentioned Sigmoid activation function layer performs Sigmoid activation on the first weight coefficient to ensure that the second weight coefficient of each channel is between 0 and 1.

[0099] Furthermore, in one embodiment of the present invention, the above-mentioned noise reduction layer can use a sparse representation decomposition method and compressed sensing technology to perform noise removal processing on the first feature image, and by converting the image features of the first feature image into a sparse basis representation, the noise is effectively filtered out, thereby further improving the accuracy of feature extraction of the second feature image.

[0100] Among them, in one embodiment of the present invention, the target feature vector and target feature map obtained through the above steps include rich information of the lesion area in the gastroscopic image to be detected.

[0101] S3, perform type diagnosis on the target feature vector through the target diagnosis model to obtain the corresponding classification result.

[0102] In one embodiment of the present invention, after obtaining the target feature vector through the above steps, the target feature vector can be diagnosed by the target diagnosis model to obtain the corresponding classification result. In one embodiment of the present invention, the target diagnosis model can include a second input layer, a third fully connected layer, a Dropout layer, a classification layer and a second output layer.

[0103] Specifically, in one embodiment of the present invention, the method of performing type diagnosis on the target feature vector through the target diagnosis model to obtain the corresponding classification result may include the following steps:

[0104] S31, the target feature vector is transmitted to the third fully connected layer through the second input layer;

[0105] S32, the third fully connected layer performs feature processing on the target feature vector to obtain a fourth feature vector;

[0106] S33, the fourth eigenvector passes through a Dropout layer to obtain a fifth eigenvector;

[0107] S34, the classification layer classifies the fifth feature vector through the Softmax function to obtain a probability vector;

[0108] S35, the second output layer model determines the category corresponding to the maximum probability value in the probability vector as the classification result and outputs it.

[0109] Among them, in one embodiment of the present invention, the above-mentioned third fully connected layer may include multiple fully connected layers (Fully Connected Layers) for further processing input features and extracting more discriminative features, and each fully connected layer is followed by a corresponding ReLU activation function to introduce nonlinearity to enhance the learning ability of the target diagnosis model.

[0110] Furthermore, in one embodiment of the present invention, the Dropout layer may randomly discard a portion of neurons during the training process to prevent overfitting and improve the generalization ability of the target diagnostic model.

[0111] Further, in one embodiment of the present invention, the classification layer classifies the fifth eigenvector by a Softmax function to obtain a probability vector, wherein the dimension of the probability vector is 4. Also, in one embodiment of the present invention, each dimension in the probability vector represents a category, and each category represents a pathological type, wherein the pathological types represented by the probability vector may include non-neoplastic lesions (gastritis and intestinal metaplasia), low-grade intraepithelial neoplasia, high-grade intraepithelial neoplasia, and early gastric cancer.

[0112] S4, based on the classification results and the target feature map, the boundary and degree of the lesion area are marked through the target boundary recognition model to obtain the corresponding target segmentation map.

[0113] In one embodiment of the present invention, after obtaining the classification result and the target feature map through the above steps, the boundary and the degree of the lesion area can be marked by the target boundary recognition model based on the classification result and the target feature map to obtain the corresponding target segmentation map. In one embodiment of the present invention, the target boundary recognition model includes a third input layer, an encoder, a skip connection layer, a decoder, a multi-scale feature fusion layer and a third output layer.

[0114] Specifically, in one embodiment of the present invention, the method for obtaining a corresponding target segmentation map by marking the boundary and the degree of the lesion area through a target boundary recognition model based on the classification result and the target feature map may include the following steps:

[0115] S41, the classification result and the target feature map are transmitted to the encoder through the third input layer;

[0116] S42, the encoder downsamples the target feature map to extract spatial features of different scales to obtain a seventh feature image;

[0117] S43, the skip connection layer transfers each layer feature of the encoder to the corresponding layer of the decoder;

[0118] S44, the decoder restores the spatial resolution of the seventh feature image layer by layer to obtain a first segmentation map corresponding to the target feature map;

[0119] S45, a multi-scale feature fusion layer fuses the first segmentation map to obtain a second segmentation map;

[0120] S46: The third output layer obtains a corresponding target segmentation map based on the classification result and the second segmentation map.

[0121] In one embodiment of the present invention, the encoder may be composed of multiple layers of convolutional layers and pooling layers, which downsample the input target feature map layer by layer, extract spatial features of different scales, and obtain the seventh feature image.

[0122] And, in one embodiment of the present invention, a skip connection layer is introduced between each layer feature of the encoder and the corresponding layer feature of the decoder to ensure that the subsequent decoder does not lose important spatial information during the decoding process.

[0123] Furthermore, in one embodiment of the present invention, the decoder may be composed of a convolution layer and an upsampling layer to restore the spatial resolution of the seventh feature image layer by layer to obtain a first segmentation map of the same size as the target feature map. After the convolution operation of each layer, an SE module is introduced to perform channel weighting on each layer of features to enhance the boundary recognition capability of the target boundary recognition model in a complex background.

[0124] Among them, in one embodiment of the present invention, the above-mentioned multi-scale feature fusion module can fuse feature maps of different levels to accurately identify lesion boundaries of different scales and capture more subtle boundary information.

[0125] Furthermore, in one embodiment of the present invention, the third output layer outputs a segmentation map of a channel corresponding to the classification result based on the classification result and the target segmentation map, wherein each channel corresponds to a type (such as chronic atrophic gastritis, intestinal metaplasia, low-grade intraepithelial neoplasia, high-grade intraepithelial neoplasia), and the boundary and degree of lesion of each lesion area can be marked in the target segmentation map.

[0126] S5, based on the target feature vector, classification result, target segmentation map and historical data of the gastroscopy image to be detected, the risk prediction result and analysis result are obtained through the target risk prediction model.

[0127] In one embodiment of the present invention, the target feature vector, classification result, and target segmentation map are obtained through the above steps, and the risk prediction result and analysis result can be obtained through the target risk prediction model based on the target feature vector, classification result, target segmentation map, and historical data of the gastroscope image to be detected. In one embodiment of the present invention, the target risk prediction model may include a fourth input layer, a time series analysis layer, a risk assessment layer, and a fourth output layer.

[0128] Specifically, in one embodiment of the present invention, the method for obtaining risk prediction results and analysis results through a target risk prediction model based on the target feature vector, classification results, target segmentation map and historical data of the gastroscopy image to be detected may include the following steps:

[0129] S51, obtaining historical data of the gastroscopic image to be detected;

[0130] S52, the target feature vector, the classification result, the target segmentation map and the historical data of the gastroscope image to be detected are transmitted to the time series analysis layer through the fourth input layer;

[0131] S53, the time series analysis layer obtains the corresponding sixth feature vector using the LSTM network through the target feature vector, the classification result, the target segmentation map and the historical data of the gastroscope image to be detected;

[0132] S54, the risk assessment layer analyzes the sixth eigenvector to obtain a predicted risk value;

[0133] S55, the fourth output layer outputs the predicted risk value as a risk prediction result;

[0134] S56, obtaining corresponding analysis results based on the risk prediction results.

[0135] In one embodiment of the present invention, the gastroscopic images to be detected may be the patient's clinical medical history, pathological report, and gastroscopic images of previous gastroscopic examinations.

[0136] Furthermore, in one embodiment of the present invention, the above-mentioned time series analysis layer can utilize an LSTM network (long short-term memory network) to process time-dependent information to capture the evolution pattern of lesions over time, and predict the future development trend of lesions through feature inputs of multiple time steps.

[0137] Furthermore, in one embodiment of the present invention, the risk assessment layer can calculate the risk of the patient progressing to high-grade intraepithelial neoplasia or gastric cancer within a preset time in the future through a fully connected layer and a logistic regression layer. The preset time can be set as needed, such as 6 months or 1 year.

[0138] Furthermore, in one embodiment of the present invention, the above risk prediction result may be a risk score to indicate the probability value of the patient developing a high-grade lesion in the future, wherein different risk scores correspond to different probability values. Furthermore, the correspondence between the risk score and the probability value may be saved in advance.

[0139] Furthermore, in one embodiment of the present invention, after obtaining the risk prediction results through the above steps, the doctor can use the above risk prediction results as the basis for clinical decision-making, formulate personalized treatment and follow-up strategies, and determine the treatment and follow-up strategies as analysis results for output.

[0140] Furthermore, in one embodiment of the present invention, before obtaining the risk prediction results and analysis results corresponding to the gastroscopic image to be detected through the above method, the above method may also include: obtaining a gastroscopic image training data set, and using the gastroscopic image training data set to train the initial feature extraction deep network, the initial diagnosis model, the initial boundary recognition model and the initial risk prediction model to obtain the target feature extraction deep network, the target diagnosis model, the target boundary recognition model and the target risk prediction model.

[0141] Among them, in one embodiment of the present invention, a gastroscopic image data set of early gastric cancer and precancerous lesions can be obtained, and the biopsy or surgical pathology report is used as the gold standard, and the gastroscopic image data in the gastroscopic image data set are annotated by senior physicians, and the annotated image data set is determined as the gastroscopic image training data set.

[0142] Also, in one embodiment of the present invention, the above-mentioned method of training the initial feature extraction deep network, the initial diagnosis model, the initial boundary recognition model and the initial risk prediction model using the gastroscopy image training data set is the same as the prior art, and the present disclosed embodiment will not be elaborated here.

[0143] The embodiment of the present invention proposes an analysis method for early gastric cancer and precancerous lesions, obtains a gastroscopic image to be detected; extracts the features of the gastroscopic image to be detected through a target feature extraction deep network, and obtains a corresponding target feature vector and a target feature map; performs type diagnosis on the target feature vector through a target diagnosis model to obtain a corresponding classification result; based on the classification result and the target feature map, the boundary of the lesion area and the degree of lesion are marked through a target boundary recognition model to obtain a corresponding target segmentation map; based on the target feature vector, the classification result, the target segmentation map and the historical data of the gastroscopic image to be detected, the risk prediction result and the analysis result are obtained through a target risk prediction model. Therefore, the analysis method for early gastric cancer and precancerous lesions proposed by the present invention can accurately identify the type of precancerous lesions and the risk of lesion progression, thereby assisting doctors in making accurate judgments on the condition, avoiding erroneous underestimation of the condition, and effectively preventing the occurrence of gastric cancer.

[0144] In order to implement the above embodiment, Figure 2 As shown, this embodiment also provides an analysis device 10 for early gastric cancer and precancerous lesions, which includes an acquisition module 100, an extraction module 200, a classification module 300, a processing module 400 and a risk prediction module 500;

[0145] An acquisition module 100 is used to acquire a gastroscopic image to be detected;

[0146] An extraction module 200 is used to extract features of the gastroscope image to be detected through a target feature extraction deep network to obtain a corresponding target feature vector and a target feature map;

[0147] The classification module 300 is used to perform type diagnosis on the target feature vector through the target diagnosis model to obtain a corresponding classification result;

[0148] The processing module 400 is used to mark the boundary and degree of the lesion area through the target boundary recognition model based on the classification result and the target feature map, and obtain the corresponding target segmentation map;

[0149] The risk prediction module 500 is used to obtain risk prediction results and analysis results through a target risk prediction model based on the target feature vector, classification results, target segmentation map and historical data of the gastroscopy image to be detected.

[0150] In one embodiment of the present invention, the target feature extraction deep network includes a first input layer, a convolutional network layer, a noise reduction layer and a first output layer; the extraction module is specifically used for:

[0151] The gastroscope image to be detected is transmitted to the convolutional network layer through the first input layer;

[0152] The convolutional network layer extracts features of the gastroscope image to be detected through multiple convolutional layers and SE modules to obtain a first feature image;

[0153] The denoising layer performs noise removal processing on the first feature image to obtain a second feature image;

[0154] The first output layer obtains the corresponding target feature vector and target feature map based on the second feature image.

[0155] In one embodiment of the present invention, the above extraction module is further used for:

[0156] The first convolutional layer is used to extract features of the gastroscope image to be detected, thereby obtaining a third feature image;

[0157] Performing feature processing on the third feature image by the first SE module to obtain a fourth feature image;

[0158] Performing feature extraction on the fourth feature image through the second convolutional layer to obtain a fifth feature image;

[0159] Performing feature processing on the fifth feature image by a second SE module to obtain a sixth feature image;

[0160] Repeat the above steps, and determine the feature image output by the last convolutional layer corresponding to the SE module as the first feature image.

[0161] In one embodiment of the present invention, the first SE module includes a global average pooling layer, a first fully connected layer, a ReLU activation function layer, a second fully connected layer, a Sigmoid activation function layer and an adaptive weighting layer; the extraction module is also used for:

[0162] The global average pooling layer performs global average pooling on the third feature image to obtain a first feature vector;

[0163] The first fully connected layer performs dimension compression on the first feature vector to obtain the second feature vector;

[0164] The ReLU activation function layer applies the ReLU activation function to the second eigenvector to obtain the third eigenvector;

[0165] The second fully connected layer expands the dimension of the third eigenvector to obtain the first weight coefficient of each channel;

[0166] The Sigmoid activation function layer performs Sigmoid activation on the first weight coefficient to obtain the second weight coefficient of each channel;

[0167] The adaptive weighting layer multiplies the second weight coefficient by the third feature image by channel to obtain a fourth feature image.

[0168] In one embodiment of the present invention, the target diagnosis model includes a second input layer, a third fully connected layer, a Dropout layer, a classification layer, and a second output layer; the classification module is specifically used for:

[0169] The target feature vector is transmitted to the third fully connected layer through the second input layer;

[0170] The third fully connected layer performs feature processing on the target feature vector to obtain the fourth feature vector;

[0171] The fourth eigenvector passes through the Dropout layer to obtain the fifth eigenvector;

[0172] The classification layer classifies the fifth feature vector through the Softmax function to obtain a probability vector;

[0173] The second output layer model determines the category corresponding to the maximum probability value in the probability vector as the classification result and outputs it.

[0174] In one embodiment of the present invention, the target boundary recognition model includes a third input layer, an encoder, a skip connection layer, a decoder, a multi-scale feature fusion layer and a third output layer; the processing module is specifically used to:

[0175] The classification result and the target feature map are transmitted to the encoder through the third input layer;

[0176] The encoder downsamples the target feature map to extract spatial features of different scales and obtains the seventh feature image;

[0177] The skip connection layer transfers the features of each layer of the encoder to the corresponding layer of the decoder;

[0178] The decoder restores the spatial resolution of the seventh feature image layer by layer to obtain a first segmentation map corresponding to the target feature map;

[0179] The multi-scale feature fusion layer fuses the first segmentation map to obtain the second segmentation map;

[0180] The third output layer obtains the corresponding target segmentation map based on the classification result and the second segmentation map.

[0181] In one embodiment of the present invention, the target risk prediction model includes a fourth input layer, a time series analysis layer, a risk assessment layer and a fourth output layer; the risk prediction result and the analysis result are obtained by the target risk prediction model based on the target feature vector, the classification result, the target segmentation map and the historical data of the gastroscope image to be detected, including:

[0182] Acquire historical data of gastroscopic images to be detected;

[0183] The target feature vector, classification result, target segmentation map and historical data of the gastroscope image to be detected are transmitted to the time series analysis layer through the fourth input layer;

[0184] The time series analysis layer uses the LSTM network to obtain the corresponding sixth eigenvector through the target feature vector, classification results, target segmentation map and historical data of the gastroscope image to be detected;

[0185] The risk assessment layer analyzes the sixth eigenvector to obtain a predicted risk value;

[0186] The fourth output layer outputs the predicted risk value as the risk prediction result;

[0187] Based on the risk prediction results, the corresponding analysis results are obtained.

[0188] According to the analysis device for early gastric cancer and precancerous lesions of the embodiment of the present invention, a gastroscopic image to be detected is obtained; the features of the gastroscopic image to be detected are extracted by a target feature extraction deep network to obtain a corresponding target feature vector and a target feature map; the target feature vector is type diagnosed by a target diagnosis model to obtain a corresponding classification result; based on the classification result and the target feature map, the boundary of the lesion area and the degree of lesion are marked by a target boundary recognition model to obtain a corresponding target segmentation map; based on the target feature vector, the classification result, the target segmentation map and the historical data of the gastroscopic image to be detected, the risk prediction result and the analysis result are obtained by a target risk prediction model. Therefore, the analysis method for early gastric cancer and precancerous lesions proposed by the present invention can accurately identify the type of precancerous lesions and the risk of lesion progression, thereby assisting doctors in making accurate judgments on the condition, avoiding erroneous underestimation of the condition, and effectively preventing the occurrence of gastric cancer.

[0189] like Figure 3 As shown, the electronic device 300 includes a computing unit 301, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 302 or a computer program loaded from a storage unit 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 can also be stored. The computing unit 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0190] A number of components in the electronic device 300 are connected to the I / O interface 305, including: an input unit 308, such as a keyboard, a mouse, etc.; an output unit 307, such as various types of displays, speakers, etc.; a storage unit 308, such as a disk, an optical disk, etc.; and a communication unit 309, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 309 allows the electronic device 300 to exchange information / data with other electronic devices through a computer network such as the Internet and / or various telecommunication networks.

[0191] The computing unit 301 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 301 performs the various methods and processes described above, such as the analysis method for early gastric cancer and precancerous lesions. For example, in some embodiments, the analysis method for early gastric cancer and precancerous lesions may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 308. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 300 via the ROM 302 and / or the communication unit 309. When the computer program is loaded into the RAM 303 and executed by the computing unit 301, one or more steps of the analysis method for early gastric cancer and precancerous lesions described above may be performed. Alternatively, in other embodiments, the computing unit 301 may be configured to execute the analysis method for early gastric cancer and precancerous lesions in any other appropriate manner (eg, by means of firmware).

[0192] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input system, and at least one output system, and transmit data and instructions to the storage system, the at least one input system, and the at least one output system.

[0193] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing system, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0194] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, system, or electronic device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or electronic device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage electronic device, a magnetic storage electronic device, or any suitable combination of the foregoing.

[0195] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display system (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing system (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of systems can also be used to provide interaction with a user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0196] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0197] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0198] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0199] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for analyzing early gastric cancer and precancerous lesions, characterized in that: The method comprises: Acquire a gastroscopic image to be detected; Extracting the features of the gastroscope image to be detected through a target feature extraction deep network to obtain a corresponding target feature vector and a target feature map; Performing type diagnosis on the target feature vector through a target diagnosis model to obtain a corresponding classification result; Based on the classification result and the target feature map, the boundary and the degree of the lesion area are marked by a target boundary recognition model to obtain a corresponding target segmentation map; Based on the target feature vector, the classification result, the target segmentation map and the historical data of the gastroscopic image to be detected, a risk prediction result and an analysis result are obtained through a target risk prediction model.

2. The method according to claim 1, characterized in that The target feature extraction deep network includes a first input layer, a convolutional network layer, a noise reduction layer and a first output layer; the feature of the gastroscope image to be detected is extracted by the target feature extraction deep network to obtain a corresponding target feature vector and a target feature map, including: The gastroscope image to be detected is transmitted to the convolutional network layer through the first input layer; The convolutional network layer extracts features of the gastroscope image to be detected through multiple convolutional layers and SE modules to obtain a first feature image; The denoising layer performs noise removal processing on the first feature image to obtain a second feature image; The first output layer obtains a corresponding target feature vector and a target feature map based on the second feature image.

3. The method according to claim 2, characterized in that Each convolutional layer is followed by a corresponding SE module; the convolutional network layer extracts features of the gastroscope image to be detected through multiple convolutional layers and SE modules to obtain a first feature image, including: Extracting features of the gastroscope image to be detected through the first convolutional layer to obtain a third feature image; Performing feature processing on the third feature image by the first SE module to obtain a fourth feature image; Performing feature extraction on the fourth feature image through a second convolutional layer to obtain a fifth feature image; Performing feature processing on the fifth feature image by a second SE module to obtain a sixth feature image; Repeat the above steps, and determine the feature image output by the last convolutional layer corresponding to the SE module as the first feature image.

4. The method according to claim 1, characterized in that The first SE module includes a global average pooling layer, a first fully connected layer, a ReLU activation function layer, a second fully connected layer, a Sigmoid activation function layer and an adaptive weighting layer; the third feature image is feature processed by the first SE module to obtain a fourth feature image, including: The global average pooling layer performs global average pooling on the third feature image to obtain a first feature vector; The first fully connected layer performs dimension compression on the first feature vector to obtain a second feature vector; The ReLU activation function layer applies a ReLU activation function to the second eigenvector to obtain a third eigenvector; The second fully connected layer performs dimension expansion on the third feature vector to obtain a first weight coefficient for each channel; The Sigmoid activation function layer performs Sigmoid activation on the first weight coefficient to obtain a second weight coefficient for each channel; The adaptive weighting layer multiplies the second weight coefficient by the third feature image by channel to obtain a fourth feature image.

5. The method according to claim 1, characterized in that The target diagnosis model includes a second input layer, a third fully connected layer, a Dropout layer, a classification layer, and a second output layer; the target diagnosis model is used to perform type diagnosis on the target feature vector to obtain a corresponding classification result, including: The target feature vector is transmitted to the third fully connected layer through the second input layer; The third fully connected layer performs feature processing on the target feature vector to obtain a fourth feature vector; The fourth eigenvector passes through the Dropout layer to obtain a fifth eigenvector; The classification layer classifies the fifth feature vector by using a Softmax function to obtain a probability vector; The second output layer determines the category corresponding to the maximum probability value in the probability vector as the classification result and outputs it.

6. The method according to claim 1, characterized in that The target boundary recognition model includes a third input layer, an encoder, a jump connection layer, a decoder, a multi-scale feature fusion layer and a third output layer; based on the classification result and the target feature map, the boundary and the degree of the lesion area are marked by the target boundary recognition model to obtain a corresponding target segmentation map, including: The classification result and the target feature map are transmitted to the encoder through the third input layer; The encoder downsamples the target feature map to extract spatial features of different scales to obtain a seventh feature image; The skip connection layer transmits each layer feature of the encoder to the corresponding layer of the decoder; The decoder restores the spatial resolution of the seventh feature image layer by layer to obtain a first segmentation map corresponding to the target feature map; The multi-scale feature fusion layer fuses the first segmentation map to obtain a second segmentation map; The third output layer obtains a corresponding target segmentation map based on the classification result and the second segmentation map.

7. The method according to claim 1, characterized in that The target risk prediction model includes a fourth input layer, a time series analysis layer, a risk assessment layer and a fourth output layer; the risk prediction result and the analysis result are obtained by the target risk prediction model based on the target feature vector, the classification result, the target segmentation map and the historical data of the gastroscope image to be detected, including: Acquiring historical data of the gastroscopic image to be detected; The target feature vector, classification result, target segmentation map and historical data of the gastroscope image to be detected are transmitted to the time series analysis layer through the fourth input layer; The time series analysis layer obtains the corresponding sixth eigenvector by using the LSTM network through the target eigenvector, the classification result, the target segmentation map and the historical data of the gastroscope image to be detected; The risk assessment layer analyzes the sixth eigenvector to obtain a predicted risk value; The fourth output layer outputs the predicted risk value as a risk prediction result; Based on the risk prediction results, corresponding analysis results are obtained.

8. An analysis device for early gastric cancer and precancerous lesions, characterized in that: The device comprises: An acquisition module, used for acquiring a gastroscopic image to be detected; An extraction module, used for extracting the features of the gastroscopic image to be detected through a target feature extraction deep network to obtain a corresponding target feature vector and a target feature map; A classification module, used to perform type diagnosis on the target feature vector through a target diagnosis model to obtain a corresponding classification result; A processing module, used for marking the boundary and degree of the lesion area through a target boundary recognition model based on the classification result and the target feature map, to obtain a corresponding target segmentation map; The risk prediction module is used to obtain risk prediction results and analysis results through a target risk prediction model based on the target feature vector, the classification result, the target segmentation map and the historical data of the gastroscopic image to be detected.

9. An electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A computer storage medium, wherein: The computer storage medium stores computer executable instructions; after the computer executable instructions are executed by the processor, the method according to any one of claims 1 to 7 can be implemented.

Images