Method and device for predicting focal areas of antrum and gastroesophagus

Abstract

The embodiment of the invention relates to a method and a device for predicting focal areas of gastric antrum and gastroesophagus. The method comprises the following steps: constructing a first segmentation model and a second segmentation model; constructing a first data set to train a first segmentation model for training; after the training is finished, constructing a second data set based on the first data set and the first segmentation model, and training a second segmentation model based on the second data set; and after training is finished, on the basis of the first segmentation model and the second segmentation model, gastric antrum and gastroesophagus focus area prediction is carried out on the stomach CT image. According to the invention, the focus area prediction accuracy can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and particularly relates to a method and device for predicting a lesion area of a gastric antrum and a gastroesophageal part. BACKGROUND

[0002] With the popularization and application of artificial intelligence technology in the medical field, some image semantic segmentation models (such as U-Net models) for lesion prediction have been applied to related fields as auxiliary tools, for example, image semantic segmentation models are used as an auxiliary tool to predict a lesion area of a computed tomography (CT) image of a gastric cancer patient. Through practice, it is found that the current traditional gastric cancer lesion segmentation scheme still has some problems: 1) The traditional scheme mostly adopts a whole stomach segmentation mechanism, and the positioning accuracy of sub-anatomical regions such as the gastric antrum and the gastroesophageal junction (referred to as the gastroesophageal part) is insufficient, and the problems of boundary blurring or region omission are prone to occur; 2) The traditional scheme does not constrain the spatial logical relationship between the whole stomach and the local (gastric antrum and gastroesophageal part), and the problem of local region boundary segmentation error is prone to occur; 3) The traditional scheme only performs lesion prediction based on the imageomic features of the CT image, and there is no prior information related to the anatomical structure to be referred to, and the problem of a high false positive / false negative ratio is prone to occur during lesion prediction, especially at the boundary position such as the gastric antrum and the gastroesophageal part. SUMMARY

[0003] The present application aims at the defects of the prior art, and provides a method and device for predicting a lesion area of a gastric antrum and a gastroesophageal part, electronic equipment and a computer readable storage medium. Two segmentation models, a first segmentation model and a second segmentation model, are designed to improve the prediction accuracy of the lesion area of the gastric antrum and the gastroesophageal part. The first segmentation model performs semantic segmentation on a whole stomach region and a gastric antrum + gastroesophageal part region of a stomach CT image input to the model, and outputs a corresponding first semantic map (i.e., a conventional whole stomach semantic map) and a second semantic map (i.e., a local gastric antrum and gastroesophageal part semantic map). The second semantic map can be further divided into two local semantic sub-maps, a gastric antrum sub-map and a gastroesophageal part sub-map. The second segmentation model adds anatomical structure prior information of the second semantic map to the stomach CT image by a feature fusion method, and performs semantic segmentation on the lesion area of the gastric antrum and the gastroesophageal part according to the fused features to obtain a third semantic map (i.e., a lesion semantic map). In order to improve the prediction accuracy of the model, three constraint losses are added to the two conventional losses when training the first segmentation model, so that the overall loss function (the first model loss function) is composed of five losses: 1) a first loss: a prediction and label loss of the first semantic map, which is used to optimize the model parameters for predicting the whole stomach semantic map; 2) a second loss: a prediction and label loss of the second semantic map, which is used to optimize the model parameters for predicting the gastric antrum and gastroesophageal part semantic map; 3) a third loss: a prediction and label loss of a difference set map = first semantic map-second semantic map (i.e., a stomach body semantic map without the gastric antrum and gastroesophageal part), which further improves the prediction accuracy of the first and second semantic maps by constraining the difference set map error; 4) a fourth loss: a prediction and label loss of an intersection map = first semantic map ∩ gastric antrum sub-map (in an ideal state, the intersection map = gastric antrum sub-map), which further improves the prediction accuracy of the first semantic map and the gastric antrum sub-map by constraining the intersection map error; and 5) a fifth loss: a prediction and label loss of an intersection map = first semantic map ∩ gastroesophageal part sub-map (in an ideal state, the intersection map = gastroesophageal part sub-map), which further improves the prediction accuracy of the first semantic map and the gastroesophageal part sub-map by constraining the intersection map error. The first segmentation model of the present application can improve the segmentation accuracy of the gastric antrum and the gastroesophageal part and reduce the segmentation error rate. The second segmentation model inputs the second semantic map output by the first segmentation model as an additional anatomical structure prior feature, which can improve the prediction accuracy of the lesion area of the gastric antrum and the gastroesophageal part and reduce the false positive / negative lesion error probability.

[0004] To achieve the above object, a first aspect of an embodiment of the present application provides a method for predicting a lesion area of a gastric antrum and a gastroesophageal part, comprising: A first segmentation model and a second segmentation model are constructed. The first segmentation model is used to perform semantic segmentation of the entire gastric region and the gastric antrum and gastroesophageal region on the gastric CT image input to the model to obtain the corresponding first semantic map and second semantic map. The second semantic map includes a semantic sub-map of the gastric antrum and a semantic sub-map of the gastroesophageal region. The second segmentation model is used to perform feature fusion on the gastric CT image input to the model and the second semantic map, and perform semantic segmentation of the lesion region of the gastric antrum and gastroesophageal region according to the fusion features to obtain the corresponding third semantic map. The model dataset for constructing the first segmentation model by collecting large amounts of CT images of gastric cancer patients is denoted as the first dataset. The first segmentation model is trained based on the first dataset; and during training, the prediction and label loss of the first semantic graph is used as the first loss, the prediction and label loss of the second semantic graph is used as the second loss, the prediction and label loss corresponding to the difference graph of the first and second semantic graphs is used as the third loss, and the prediction and label loss of the two intersection graphs corresponding to the two sub-semantic graphs of the first semantic graph and the second semantic graph are used as the fourth and fifth losses, and the corresponding model training loss function is constructed based on the first, second, third, fourth and fifth losses; After the first segmentation model is trained, the model dataset for constructing the second segmentation model based on the first dataset and the first segmentation model is denoted as the second dataset. The second segmentation model is trained based on the second dataset; After the second segmentation model is trained, the lesion areas in the gastric antrum and gastroesophageal region are predicted based on the first and second segmentation models of the gastric CT image input by the user, and the prediction results are fed back to the current user.

[0005] Preferably, the model input terminal of the first segmentation model is used to receive the gastric CT image, the first model output terminal is used to output the corresponding first semantic map, and the second model output terminal is used to output the corresponding second semantic map. The shape of the gastric CT image is D0×H0×W0, where D0, H0, and W0 are the preset image feature dimensions, image height, and image width, respectively; the gastric CT image is composed of H0×W0 image pixels, and each image pixel includes D0 image features. The shape of the first semantic map is D1×H0×W0, where D1 is a preset first feature dimension and D1=1; the first semantic map is composed of H0×W0 first pixels; each first pixel includes a first semantic feature; the first semantic feature is a binary feature, including 0 and 1; when the first semantic feature is 0, it indicates that the current pixel is not in the whole stomach region, and when it is 1, it indicates that the current pixel is in the whole stomach region; The graph shape of the second semantic graph is D2xH0xW0, D2 is a preset second feature dimension, and D2=2; the second semantic graph is composed of H0xW0 second pixel points, each of the second pixel points includes two semantic features, namely a second semantic feature and a third semantic feature; the second and third semantic features are each a binary feature including 0 and 1; the feature combination of the second and third semantic features has only three combination modes, namely [0, 0], [0, 1] and [1, 0]; when the feature combination of the second and third semantic features is [0, 0], it indicates that the current pixel point is neither in the antral region nor in the gastroesophageal region, when the feature combination is [1, 0], it indicates that the current pixel point is in the antral region, and when the feature combination is [0, 1], it indicates that the current pixel point is in the gastroesophageal region; H0xW0 second semantic features of all the second pixel points form a corresponding antral semantic subgraph, and H0xW0 third semantic features form a corresponding gastroesophageal semantic subgraph; The first segmentation model includes a first U-Net model, a first CNN layer and a second CNN layer; The input end of the first U-Net model is connected with the model input end, and the output end is connected with the input ends of the first CNN layer and the second CNN layer respectively; the output ends of the first CNN layer and the second CNN layer are connected with the first model output end and the second model output end respectively; The first U-Net model is realized based on a conventional U-Net model structure, but does not include a last 1x1 convolution layer of the conventional U-Net model; the first U-Net model is used for feature extraction processing of the gastric CT image to obtain a corresponding feature tensor H1 which is sent to the first CNN layer and the second CNN layer; the shape of the feature tensor H1 is D3xH0xW0, D3 is a preset third feature dimension, and D3>D2; The first CNN layer is used for convolution operation on the feature tensor H1 using a convolution kernel with a shape of D3x1x1 to obtain a corresponding first semantic graph; The second CNN layer is used for convolution operation on the feature tensor H1 using two convolution kernels with a shape of D3x1x1 to obtain a corresponding antral semantic subgraph and a corresponding gastroesophageal semantic subgraph to form a corresponding second semantic graph.

[0006] Preferably, the first model input end and the second model input end of the second segmentation model are used for receiving the corresponding gastric CT image and the second semantic graph, and the model output end is used for outputting the corresponding third semantic graph; The third semantic graph has a graph shape of D4xH0xW0, D4 is a preset fourth feature dimension, and D4=1; the third semantic graph is composed of H0xW0 third pixel points; each third pixel point includes a fourth semantic feature; the fourth semantic feature is a binary feature including 0 and 1; when the fourth semantic feature is 0, it indicates that the current pixel point is not in the gastric cancer lesion area, and when the fourth semantic feature is 1, it indicates that the current pixel point is in the gastric cancer lesion area; The second segmentation model includes a feature fusion module, a second U-Net model and a third CNN layer. The first and second input ends of the feature fusion module are connected with the first model input end and the second model input end respectively, and the output end is connected with the input end of the second U-Net model; the output end of the second U-Net model is connected with the input end of the third CNN layer; and the output end of the third CNN layer is connected with the model output end. The feature fusion module is used for pixel-level feature fusion processing of the gastric CT image and the second semantic graph to obtain a corresponding fusion feature map, which is sent to the second U-Net model; the fusion feature map has a graph shape of D5xH0xW0, D4 is a preset fifth feature dimension, and D5=D0+D2; the fusion feature map is composed of H0xW0 fourth pixel points; each fourth pixel point includes a first fusion feature; the first fusion feature is composed of corresponding D0 image features, the second semantic feature and the third semantic feature; The second U-Net model is based on a conventional U-Net model structure, but does not include the last 1x1 convolution layer of the conventional U-Net model; the second U-Net model is used for feature extraction processing of the fusion feature map to obtain a corresponding feature tensor H2, which is sent to the third CNN layer; the feature tensor H2 has a shape of D6xH0xW0, D6 is a preset sixth feature dimension, and D6>D4; The third CNN layer is used for convolution operation of the feature tensor H1 using a convolution kernel with a shape of D6x1x1 to obtain the corresponding third semantic graph.

[0007] Preferably, the first data set includes a plurality of first data records; the first data record includes a first training image, a first label map and a second label map; The first training image is a CT image of the stomach of a gastric cancer patient; the first label map has a graph shape of D1xH0xW0; the first label map is a first semantic map labeled with semantic feature labels by manual labeling or other machine labeling; the second label map has a graph shape of D2xH0xW0; the second label map is a second semantic map labeled with semantic feature labels by manual labeling or other machine labeling; two semantic subgraphs of the second label map are denoted as a gastric antrum label subgraph and a gastroesophageal label subgraph; The second data set includes a plurality of second data records; the second data records include the first training image, the first training semantic map, and a third label map; The first training semantic map has a graph shape of D2xH0xW0; the first training semantic map is a second semantic map; two semantic subgraphs of the first training semantic map are denoted as a gastric antrum training subgraph and a gastroesophageal training subgraph; the third label map has a graph shape of D4xH0xW0; the third label map is a third semantic map labeled with semantic feature labels by manual labeling or other machine labeling.

[0008] Preferably, the model data set of the first segmentation model constructed by collecting big data of CT images of gastric cancer patients is denoted as a first data set, and specifically includes: Step 51, collecting big data of CT images of a default scanning plane of a gastric cancer patient through a plurality of public data collection channels to obtain a corresponding original image set; The public data collection channels include public medical image databases and public medical literature / paper databases; the default scanning plane is a coronal plane; and the original image set includes a plurality of original CT images. Step 52, identifying whether the height, width, and feature dimension of each original CT image of the original image set match a reference image standard based on the graph shape of the CT image of the stomach; if they match, the current original CT image is taken as a corresponding preprocessed image; if they do not match, the height and width of the current original CT image are scaled or cropped based on the reference image standard to ensure that the graph size matches, and the feature dimension of the current original CT image is one-to-one mapped or aligned with the feature dimension of the reference image standard to ensure that the feature dimensions match, and the original CT image after the processing is taken as a corresponding preprocessed image; Step 53, taking each of the obtained preprocessed images as a corresponding current image; taking the current image as a corresponding first training image; and labeling the current image with a corresponding first label map in the data format of the first semantic map by manual labeling or other machine labeling; labeling the current image with a corresponding second label map in the data format of the second semantic map by manual labeling or other machine labeling; and taking the first training image, the first label map and the second label map corresponding to the current image to form a corresponding first data record; Step 54, taking all the obtained first data records to form a corresponding first data set.

[0009] Preferably, the training of the first segmentation model based on the first data set specifically includes: Step 61, randomly segmenting the first data set into two sub-data sets according to a preset first segmentation ratio, denoted as a first training set and a first evaluation set; Wherein, the first training set and the first evaluation set are both composed of a plurality of first data records; the total number of records of the first training set and the first evaluation set satisfies the first segmentation ratio; Step 62, taking the first first data record of the first training set as a corresponding current training record; Step 63, taking the first training image of the current training record as the current gastric CT image to input the first segmentation model for processing, and taking the first semantic map and the second semantic map obtained by this processing as corresponding prediction maps and prediction maps ; and taking the antral semantic subgraph and the gastroesophageal semantic subgraph of the prediction map as corresponding prediction subgraphs and prediction subgraphs ; and taking the first label map and the second label map of the current training record as corresponding label maps and label maps ; and taking the antral semantic subgraph and the gastroesophageal semantic subgraph of the label map as corresponding label subgraphs and label subgraphs ; Step 64, taking the difference set of the prediction map and the prediction map as a corresponding difference set ; and taking the difference set of the label map and the label map as a corresponding difference set ; and the predicted graph and the predicted subgraph The intersection graph is denoted as the corresponding intersection graph. ⊙ represents the Hadamard product; and the predicted graph is... and the predicted subgraph The intersection graph is denoted as the corresponding intersection graph. ; and the label image and the label subgraph The intersection graph is denoted as the corresponding intersection graph. ; and the label image and the label subgraph The intersection graph is denoted as the corresponding intersection graph. ; Step 65, from the predicted graph and the label image Form a corresponding first prediction-label pair ( , ), from the predicted graph and the label image Form a corresponding second prediction-label pair ( , ), from the difference graph and the difference graph Form a corresponding third prediction-label pair ( , ), from the intersection graph and the intersection graph Form a corresponding fourth prediction-label pair ( , ), from the intersection graph and the intersection graph Form a corresponding fifth prediction-label pair ( , ); Step 66: The prediction and label loss of the first semantic graph is used as the first loss; the prediction and label loss of the second semantic graph is used as the second loss; the prediction and label loss corresponding to the difference graph of the first and second semantic graphs is used as the third loss; and the prediction and label loss of the two intersection graphs corresponding to the two sub-semantic graphs of the first and second semantic graphs are used as the fourth and fifth losses. Based on the first, second, third, fourth, and fifth losses, a corresponding first model loss function L is constructed. M1 The first, second, third, fourth, and fifth prediction-label pairs are then input into the first model's loss function L. M1 The corresponding first loss value is obtained through calculation; Wherein, the first model loss function LM1 is: , , , , , ; L1, L2, L3, L4, L5 are corresponding first, second, third, fourth and fifth losses; the first loss is used for parameter optimization of the first U-Net model and the first CNN layer; the second loss is used for parameter optimization of the first U-Net model and the second CNN layer; the third, fourth and fifth losses are all used for optimization of the first U-Net model and mutual constraint of the first CNN layer and the second CNN layer; w1, w2, w3, w4, w5 are corresponding first, second, third, fourth and fifth loss weights; L CE () is a cross-entropy loss function; Step 67, whether the first loss value meets a preset first loss value range is identified; if the first loss value meets the first loss value range, whether the current training record is the last first data record of the first training set is identified, if yes, it is turned to step 68, if not, the next first data record of the first training set is extracted as a new current training record and it is returned to step 63; if the first loss value does not meet the first loss value range, a first segmentation model is subjected to a round of parameter modulation based on a preset first model optimizer towards a direction of making the first loss L1, the second loss L2, the third loss L3, the fourth loss L4, the fifth loss L5 and the first model loss function L M1 all reach a minimum value, and it is returned to step 63 at the end of the round of parameter modulation; Among them, the first model optimizer includes an Adam optimizer, an SGD optimizer; Step 68, a round of traversal is performed on all the first data records of the first evaluation set; and during the round of traversal, the first data record currently traversed is taken as a corresponding current evaluation record; the first training image of the current evaluation record is taken as the current stomach CT image, and the first segmentation model is inputted for processing, and the first semantic graph and the second semantic graph obtained by this processing are taken as corresponding first prediction graph and second prediction graph; and a corresponding sixth prediction-label pair is formed by the first prediction graph and the first label graph of the current evaluation record, and a corresponding seventh prediction-label pair is formed by the second prediction graph and the second label graph of the current evaluation record; and when the round of traversal ends, the first F1 score is obtained by F1 score evaluation according to all the sixth prediction-label pairs obtained in the round of traversal; and the second F1 score is obtained by F1 score evaluation according to all the seventh prediction-label pairs obtained. Step 69, whether the first F1 score and the second F1 score both satisfy the respective first F1 score range and second F1 score range is identified; if not, step 61 is returned to continue training; if yes, it is confirmed that the model training of the first segmentation model is ended.

[0010] Preferably, the model data set for constructing the second segmentation model based on the first data set and the first segmentation model is recorded as a second data set, and specifically includes: Each first data record of the first data set is taken as a corresponding current data record; and the first training image of the current data record is taken as the current stomach CT image, and the first segmentation model is inputted for processing to obtain the corresponding first semantic graph and second semantic graph; and the second semantic graph obtained this time is taken as a corresponding first training semantic graph; and a corresponding third label graph is labeled for the first training image of the current data record in the data format of the third semantic graph by manual labeling or other machine labeling; and a corresponding second data record is formed by the first training image, the first training semantic graph and the third label graph of the current data record; and all the second data records obtained are taken as the corresponding second data set.

[0011] Preferably, the second segmentation model is trained based on the second data set, and specifically includes: Step 81, the second data set is randomly divided into two sub-data sets recorded as a second training set and a second evaluation set based on a preset second segmentation ratio; The second training set and the second evaluation set are both composed of a plurality of second data records; the total number ratio of the second training set and the second evaluation set satisfies the second segmentation ratio. Step 82, taking the first second data record of the second training set as a corresponding current training record; Step 83, taking the first training image and the first training semantic graph of the current training record as the current stomach CT image and the second semantic graph to input the second segmentation model for processing, and taking the third semantic graph obtained by this processing as a corresponding prediction graph ; and taking the third label graph of the current training record as a corresponding label graph ; and the prediction graph and the label graph comprise a corresponding eighth prediction-label loss pair , ); Step 84, taking the eighth prediction-label loss pair , ) into a preset second model loss function L M2 to calculate a corresponding second loss value; Wherein, the second model loss function L M2 is: ; L CE () is a cross-entropy loss function; Step 85, identifying whether the second loss value meets a preset second loss value range; if the second loss value meets the second loss value range, identifying whether the current training record is the last second data record of the second training set, if yes, going to step 86, if no, extracting the next second data record of the second training set as a new current training record and returning to step 83; if the second loss value does not meet the second loss value range, performing a round of parameter modulation on the second segmentation model based on a preset second model optimizer towards a direction that makes the second model loss function L M2 reach a minimum value, and returning to step 83 at the end of this round of parameter modulation; Wherein, the second model optimizer includes an Adam optimizer, an SGD optimizer; Step 86, a round of traversal is performed on all the second data records of the second evaluation set; and during the round of traversal, the second data record currently traversed is taken as a corresponding current evaluation record; the first training image and the first training semantic graph of the current evaluation record are taken as the current gastric CT image and the second semantic graph, and are input into the second segmentation model for processing, and the third semantic graph obtained by this processing is recorded as a corresponding third prediction graph; and the third label graph of the current evaluation record is recorded as a corresponding third prediction graph; and a corresponding ninth prediction-label pair is formed by the third prediction graph and the third label graph of the current evaluation record; and when the round of traversal ends, a third F1 score is obtained by performing F1 score evaluation on all the ninth prediction-label pairs obtained by the round of traversal; Step 87, whether the third F1 score meets a preset third F1 score range is identified; if not, step 81 is returned to continue training; if yes, it is confirmed that the model training of the second segmentation model ends.

[0012] Preferably, the gastric antrum and gastroesophageal part lesion area prediction method based on the first and second segmentation models predicts the lesion area of the gastric antrum and gastroesophageal part of the gastric CT image input by the user and feeds back the prediction result to the current user, and specifically comprises: The gastric CT image input by the user is input into the first segmentation model for processing to obtain the corresponding first semantic graph and second semantic graph; and the current gastric CT image and the second semantic graph are input into the second segmentation model for processing to obtain the corresponding third semantic graph; and the gastric CT image is labeled based on the third semantic graph; and the labeled gastric CT image is fed back to the current user as the current prediction result.

[0013] The second aspect of the embodiment of the application provides a device for implementing the gastric antrum and gastroesophageal part lesion area prediction method of the first aspect, and the device comprises a model construction module, a first data preparation module, a first model training module, a second data preparation module, a second model training module and a model application module. The model construction module is used to construct a first segmentation model and a second segmentation model; the first segmentation model is used to perform semantic segmentation on a gastric CT image input by the model to obtain a corresponding first semantic graph and second semantic graph, the second semantic graph comprising a gastric antrum semantic subgraph and a gastroesophageal part semantic subgraph; and the second segmentation model is used to perform feature fusion on the gastric CT image input by the model and the second semantic graph, and perform semantic segmentation on the lesion area of the gastric antrum and gastroesophageal part according to the fused features to obtain a corresponding third semantic graph; The first data preparation module is configured to construct a model data set of the first segmentation model by collecting big data from CT images of gastric cancer patients, which is referred to as a first data set. The first model training module is configured to train the first segmentation model based on the first data set, and during the training, the first model training module is configured to take the prediction loss of the first semantic graph as a first loss, take the prediction loss of the second semantic graph as a second loss, take the prediction loss of the difference set graph of the first semantic graph and the second semantic graph as a third loss, take the prediction loss of the two intersection graphs corresponding to the two sub-semantic graphs of the first semantic graph and the second semantic graph as a fourth loss and a fifth loss, and construct a corresponding model training loss function based on the first, second, third, fourth and fifth losses. The second data preparation module is configured to construct a model data set of the second segmentation model based on the first data set and the first segmentation model after the training of the first segmentation model is completed, which is referred to as a second data set. The second model training module is configured to train the second segmentation model based on the second data set. The model application module is configured to perform lesion area prediction of the antrum and the gastroesophageal part based on the first and second segmentation models after the training of the second segmentation model is completed, and feed back the prediction result to the current user.

[0014] The third aspect of the embodiment of the present application provides an electronic device, which comprises a memory, a processor and a transceiver. The processor is configured to be coupled with the memory, read and execute instructions in the memory, so as to realize the method steps of the first aspect. The transceiver is coupled with the processor, and the transceiver is controlled by the processor to perform message transceiving.

[0015] The fourth aspect of the embodiment of the present application provides a computer readable storage medium, which stores computer instructions, when the computer instructions are executed by a computer, the computer instructions make the computer execute the instructions of the method of the first aspect.

[0016] The embodiment of the present application provides a gastric antrum and a lesion area prediction method, device, electronic equipment and computer readable storage medium of a gastroesophageal part. From the above content, it can be known that two segmentation models, a first segmentation model and a second segmentation model, are designed to improve the prediction accuracy of the lesion area of the gastric antrum and the gastroesophageal part. The first segmentation model performs full stomach region and gastric antrum+gastroesophageal region semantic segmentation on the stomach CT image input by the model to output a corresponding first semantic graph (i.e. a conventional full stomach semantic graph) and a second semantic graph (i.e. a local gastric antrum and gastroesophageal semantic graph), and the second semantic graph can be further divided into two local semantic subgraphs: a gastric antrum part and a gastroesophageal part semantic subgraph. The second segmentation model adds the anatomical structure prior information of the second semantic graph to the stomach CT image by a feature fusion manner and performs semantic segmentation on the lesion area of the gastric antrum and the gastroesophageal part according to the fused features to obtain a third semantic graph (i.e. a lesion semantic graph). In order to improve the prediction accuracy of the model, three types of constraint losses are added to the two types of conventional losses when training the first segmentation model, so that the overall loss function (the first model loss function) is composed of five types of losses: 1) the first loss: the prediction and label loss of the first semantic graph, which is used to optimize the model parameters for predicting the full stomach semantic graph; 2) the second loss: the prediction and label loss of the second semantic graph, which is used to optimize the model parameters for predicting the gastric antrum and gastroesophageal semantic graph; 3) the third loss: the prediction and label loss of the difference set graph = the first semantic graph-second semantic graph (i.e. the stomach body semantic graph without the gastric antrum and gastroesophageal part), which further improves the prediction accuracy of the first and second semantic graphs by constraining the difference set graph error; 4) the fourth loss: the prediction and label loss of the intersection graph = the first semantic graph intersection gastric antrum part semantic subgraph (in an ideal state, the intersection graph = the gastric antrum part semantic subgraph), which further improves the prediction accuracy of the first semantic graph and the gastric antrum part semantic subgraph by constraining the intersection graph error; 4) the fifth loss: the prediction and label loss of the intersection graph = the first semantic graph intersection gastroesophageal part semantic subgraph (in an ideal state, the intersection graph = the gastroesophageal part semantic subgraph), which further improves the prediction accuracy of the first semantic graph and the gastroesophageal part semantic subgraph by constraining the intersection graph error. The first segmentation model of the embodiment of the present application improves the segmentation accuracy of the gastric antrum part and the gastroesophageal part and reduces the segmentation error rate. The second semantic graph output by the first segmentation model is input into the second segmentation model as an additional anatomical structure prior feature, which improves the prediction accuracy of the lesion area of the gastric antrum and the gastroesophageal part and reduces the false positive / false negative error probability. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 A gastric antrum and a lesion area prediction method schematic diagram is provided for the embodiment one of the present application. Figure 2 A full stomach, gastric antrum part and gastroesophageal part schematic diagram is provided for the embodiment one of the present application. Figure 3 A module schematic diagram of the first segmentation model provided for the embodiment one of the present application; Figure 4 A schematic diagram of the first semantic graph and the second semantic graph provided for the embodiment one of the present application; Figure 5 A module schematic diagram of the second segmentation model provided for the embodiment one of the present application; Figure 6 A module structure diagram of a lesion area prediction device for the antrum and the gastroesophageal part provided for the embodiment two of the present application; Figure 7 A structure schematic diagram of an electronic device provided for the embodiment three of the present application. DETAILED DESCRIPTION

[0018] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0019] The embodiment one of the present application provides a lesion area prediction method for the antrum and the gastroesophageal part, which comprises the following steps: Figure 1 A lesion area prediction method for the antrum and the gastroesophageal part provided for the embodiment one of the present application is shown in a schematic diagram, and the method mainly comprises the following steps: Step 1, constructing a first segmentation model and a second segmentation model.

[0020] Here, the whole stomach region, the antrum region and the gastroesophageal part region mentioned in the embodiment of the present application are as shown in a schematic diagram provided for the embodiment one of the present application. Figure 2 A schematic diagram of the whole stomach, the antrum part and the gastroesophageal part provided for the embodiment one of the present application is shown. Among them, the antrum region is included in the whole stomach region, and the gastroesophageal part region is included in the whole stomach region. The antrum region is the connecting region of the stomach and the duodenum, and the gastroesophageal part region is the connecting region of the stomach and the esophagus (also known as the esophagus).

[0021] The first segmentation model of the embodiment of the present application is used for performing semantic segmentation of the whole stomach region and the antrum and the gastroesophageal part region on the stomach CT image input to the model to obtain corresponding first semantic graph and second semantic graph. Figure 3 A module schematic diagram of the first segmentation model provided for the embodiment one of the present application is shown.

[0022] The gastric CT image of this embodiment has a shape of D0×H0×W0, where D0, H0, and W0 are the preset image feature dimensions, image height, and image width, respectively. The gastric CT image consists of H0×W0 image pixels, and each image pixel includes D0 image features. It should be noted that the gastric CT images of this embodiment are all coronal CT images.

[0023] The first semantic map in this embodiment of the invention is actually the whole stomach semantic map, such as Figure 4 The diagram illustrates the first semantic map and the second semantic map provided in Embodiment 1 of the present invention. The shape of the first semantic map is D1×H0×W0, where D1 is a preset first feature dimension, and D1=1. The first semantic map consists of H0×W0 first pixels. Each first pixel includes a first semantic feature. The first semantic feature is a binary feature, including 0 and 1. When the first semantic feature is 0, it indicates that the current pixel is not in the entire stomach region; when it is 1, it indicates that the current pixel is in the entire stomach region.

[0024] The second semantic map in this embodiment of the invention is actually a local semantic map of the gastric antrum and gastroesophageal region, such as... Figure 4 As shown, the second semantic map consists of a semantic sub-map of the gastric antrum and a semantic sub-map of the gastroesophageal region. The shape of the second semantic map is D2×H0×W0, where D2 is the preset second feature dimension, D2=2. The second semantic map consists of H0×W0 second pixels, each of which includes two semantic features: a second semantic feature and a third semantic feature. The second and third semantic features are each a binary feature, including 0 and 1. The feature combinations of the second and third semantic features have only three possible combinations: [0,0], [0,1], and [1,0]. When the feature combination of the second and third semantic features is [0,0], it indicates that the current pixel is neither in the gastric antrum nor in the gastroesophageal region; when it is [1,0], it indicates that the current pixel is in the gastric antrum; and when it is [0,1], it indicates that the current pixel is in the gastroesophageal region. The H0×W0 second semantic features of all the second pixels form the corresponding semantic sub-map of the gastric antrum, and the H0×W0 third semantic features form the corresponding semantic sub-map of the gastroesophageal region.

[0025] like Figure 3 As shown, the input end of the first segmentation model is used to receive gastric CT images, the output end of the first model is used to output the corresponding first semantic map, and the output end of the second model is used to output the corresponding second semantic map.

[0026] The model components of the first segmentation model include: a first U-Net model, a first CNN layer, and a second CNN layer.

[0027] The model component connection relationship of the first segmentation model is: the input end of the first U-Net model is connected with the model input end, the output end is connected with the input end of the first CNN layer and the second CNN layer respectively; the output end of the first CNN layer and the second CNN layer is connected with the first model output end and the second model output end respectively.

[0028] The model component functions of the first segmentation model are as follows.

[0029] 1) The first U-Net model: The first U-Net model is realized based on the conventional U-Net model structure, but does not contain the last 1*1 convolution layer of the conventional U-Net model.

[0030] The first U-Net model is used for feature extraction processing of the stomach CT image to obtain the corresponding feature tensor H1, which is sent to the first CNN layer and the second CNN layer. The shape of the feature tensor H1 is D3*H0*W0, D3 is a preset third feature dimension, and D3>D2.

[0031] 2) The first CNN layer: The first CNN layer is used for convolution operation on the feature tensor H1 using a 1*1 convolution kernel with a shape of D3*1*1 to obtain the corresponding first semantic graph.

[0032] 3) The second CNN layer: The second CNN layer is used for convolution operation on the feature tensor H1 using two 1*1 convolution kernels with a shape of D3*1*1 to obtain the corresponding antral semantic subgraph and esophagogastric semantic subgraph to form the corresponding second semantic graph.

[0033] The second segmentation model of the embodiment of the application is used for feature fusion of the model input stomach CT image and the second semantic graph, and performs semantic segmentation on the lesion area of the antrum and the esophagogastric part according to the fused features to obtain the corresponding third semantic graph, as shown in the module schematic view of the second segmentation model provided by the embodiment of the application. Figure 5 The first model input end and the second model input end of the second segmentation model are used for receiving the corresponding stomach CT image and the second semantic graph, and the model output end is used for outputting the corresponding third semantic graph.

[0034] As shown in the module schematic view of the second segmentation model provided by the embodiment of the application. Figure 5

[0035] ​The third semantic graph of the embodiment of the application has a graph shape of D4xH0xW0, D4 is a preset fourth feature dimension, and D4=1; the third semantic graph is composed of H0xW0 third pixel points; each third pixel point includes a fourth semantic feature; the fourth semantic feature is a binary feature including 0 and 1; the fourth semantic feature is 0, indicating that the current pixel point is not in the gastric cancer lesion area, and is 1, indicating that the current pixel point is in the gastric cancer lesion area.

[0036] The model components of the second segmentation model include: a feature fusion module, a second U-Net model and a third CNN layer.

[0037] The connection relationship of the model components of the second segmentation model is as follows: the first and second input ends of the feature fusion module are connected with the first model input end and the second model input end respectively, and the output end is connected with the input end of the second U-Net model; the output end of the second U-Net model is connected with the input end of the third CNN layer; and the output end of the third CNN layer is connected with the model output end.

[0038] The functions of the model components of the second segmentation model are as follows.

[0039] 1) Feature fusion module: The feature fusion module is used for performing pixel-level feature fusion processing on the gastric CT image and the second semantic graph to obtain a corresponding fusion feature graph, which is sent to the second U-Net model.

[0040] The fusion feature graph has a graph shape of D5xH0xW0, D4 is a preset fifth feature dimension, and D5=D0+D2; the fusion feature graph is composed of H0xW0 fourth pixel points; each fourth pixel point includes a first fusion feature; and the first fusion feature is composed of corresponding D0 image features, second semantic features and third semantic features.

[0041] 2) Second U-Net model: The second U-Net model is realized based on a conventional U-Net model structure, but does not include the last 1x1 convolution layer of the conventional U-Net model.

[0042] The second U-Net model is used for performing feature extraction processing on the fusion feature graph to obtain a corresponding feature tensor H2, which is sent to the third CNN layer. The feature tensor H2 has a shape of D6xH0xW0, D6 is a preset sixth feature dimension, and D6>D4.

[0043] 3) Third CNN layer: The third CNN layer is used for performing convolution operation on the feature tensor H1 using a 1x1 convolution kernel with a shape of D6x1x1 to obtain a corresponding third semantic graph.

[0044] Step 2, a first data set is constructed by collecting big data from CT images of gastric cancer patients to construct a model data set of the first segmentation model, denoted as the first data set; The first data set includes a plurality of first data records; the first data record includes a first training image, a first label map and a second label map; the first training image has a graph shape of D0xH0xW0; the first training image is a gastric CT image of a gastric cancer patient; the first label map has a graph shape of D1xH0xW0; the first label map is a first semantic map labeled by artificial labeling or other machine labeling; the second label map has a graph shape of D2xH0xW0; the second label map is a second semantic map labeled by artificial labeling or other machine labeling; two semantic subgraphs of the second label map are denoted as a gastric antrum label subgraph and a gastroesophageal label subgraph; Specifically, step 21, a plurality of public data collection channels are used to collect CT images of a gastric cancer patient on a default scanning plane to obtain a corresponding original image set; The public data collection channels include a public medical image database and a public medical literature / paper database; the default scanning plane is the coronal plane; and the original image set includes a plurality of original CT images; Step 22, whether the height, width and feature dimension of each original CT image of the original image set match the reference image standard is identified based on the graph shape of the gastric CT image as the reference image standard; if they match, the current original CT image is taken as a corresponding preprocessed image; if they do not match, the height and width of the current original CT image are scaled or cropped based on the reference image standard to ensure that the image size matches, and the feature dimension of the current original CT image is one-to-one mapped or aligned with the feature dimension of the reference image standard to ensure that the feature dimension matches, and the original CT image after the processing is taken as a corresponding preprocessed image; Step 23, each preprocessed image obtained is taken as a corresponding current image; the current image is taken as a corresponding first training image; a corresponding first label map is labeled for the current image by artificial labeling or other machine labeling according to the data format of the first semantic map; a corresponding second label map is labeled for the current image by artificial labeling or other machine labeling according to the data format of the second semantic map; and a corresponding first data record is composed of the first training image, the first label map and the second label map corresponding to the current image; Step 24, the first data set is composed of all the first data records obtained.

[0045] Step 3, the first segmentation model is trained based on the first data set. Here, the embodiment of the present application takes the prediction and label loss of the first semantic graph as the first loss, takes the prediction and label loss of the second semantic graph as the second loss, takes the prediction and label loss of the difference set graph corresponding to the first and second semantic graphs as the third loss, takes the prediction and label loss of the two intersection graphs corresponding to the two sub-semantic graphs of the first and second semantic graphs as the fourth and fifth losses during training, and constructs a corresponding model training loss function based on the first, second, third, fourth and fifth losses. Specifically comprising: step 31, based on a preset first segmentation ratio, randomly segmenting the first data set into two sub-data sets, denoted as a first training set and a first evaluation set; Wherein, the first segmentation ratio is a pre-set ratio parameter, for example, 8:2; the first training set and the first evaluation set are both composed of a plurality of first data records; the total number of records of the first training set and the first evaluation set satisfies the first segmentation ratio; Step 32, taking the first first data record of the first training set as the corresponding current training record; Step 33, taking the first training image of the current training record as the current gastric CT image, inputting the first segmentation model for processing, and taking the first semantic graph and the second semantic graph obtained by this processing as the corresponding prediction graph And prediction graph ; and taking the antral semantic sub-graph and the gastroesophageal semantic sub-graph of the prediction graph as the corresponding prediction sub-graph And prediction sub-graph ; and taking the first label graph and the second label graph of the current training record as the corresponding label graph And label graph ; and taking the antral semantic sub-graph and the gastroesophageal semantic sub-graph of the label graph as the corresponding label sub-graph And label sub-graph ; Step 34, taking the difference set graph of the prediction graph And prediction graph as the corresponding difference set graph ; and taking the difference set graph of the label graph And label graph as the corresponding difference set graph ; and taking the intersection graph of the prediction graph And prediction sub-graph as the corresponding intersection graph , is the Hadamard product; and taking the intersection graph of the prediction graph And prediction sub-graph as the corresponding intersection graph ; and taking the intersection graph of the label graph And label sub-graph The intersection graph of the prediction graph and the label graph is denoted as the corresponding intersection graph The intersection graph of the prediction graph and the label graph is denoted as the corresponding intersection graph ; Step 35, a corresponding first prediction-label pair ( , ) is composed of the prediction graph and the label graph , a corresponding second prediction-label pair ( , ) is composed of the prediction graph and the label graph , a corresponding third prediction-label pair ( , ) is composed of the difference set graph and the difference set graph , a corresponding fourth prediction-label pair ( , ) is composed of the intersection graph and the intersection graph , and a corresponding fifth prediction-label pair ( , ) is composed of the intersection graph and the intersection graph ; Step 36, the prediction and label loss of the first semantic graph is taken as a first loss, the prediction and label loss of the second semantic graph is taken as a second loss, the prediction and label loss of the difference set graph corresponding to the first and second semantic graphs is taken as a third loss, the prediction and label loss of the two intersection graphs corresponding to the two sub-semantic graphs of the first and second semantic graphs is taken as a fourth and fifth loss, and a corresponding first model loss function L M1 is constructed based on the first, second, third, fourth and fifth losses M1 ; Here, the first model loss function L M1 of the embodiment of the application is as follows: , , , , , ; Wherein, L1, L2, L3, L4, L5 are corresponding first, second, third, fourth and fifth losses; the first loss is used for parameter optimization of the first U-Net model and the first CNN layer; the second loss is used for parameter optimization of the first U-Net model and the second CNN layer; the third, fourth and fifth losses are all used for optimization of the first U-Net model and mutual constraint of the first CNN layer and the second CNN layer; w1, w2, w3, w4, w5 are corresponding first, second, third, fourth and fifth loss weights; L CE () is a cross-entropy loss function; Step 37, whether the first loss value meets the preset first loss value range is identified; if the first loss value meets the first loss value range, whether the current training record is the last first data record of the first training set is identified, if yes, it is transferred to step 38, if no, the next first data record of the first training set is extracted as a new current training record and returns to step 33; if the first loss value does not meet the first loss value range, a round of parameter modulation is performed on the first segmentation model based on the preset first model optimizer towards the direction of making the first loss L1, the second loss L2, the third loss L3, the fourth loss L4, the fifth loss L5 and the first model loss function L M1 all reach the minimum value, and returns to step 33 at the end of the round of parameter modulation; Wherein, the first loss value range is a pre-set numerical range; the first model optimizer includes an Adam optimizer, an SGD optimizer; Step 38, all first data records of the first evaluation set are traversed in a round; and in the round of traversal, the first data record currently traversed is taken as a corresponding current evaluation record; the first training image of the current evaluation record is taken as a current stomach CT image and input into the first segmentation model for processing, and the first semantic map and the second semantic map obtained by this processing are taken as corresponding first prediction map and second prediction map; and the first prediction map and the first label map of the current evaluation record form a corresponding sixth prediction-label pair, and the second prediction map and the second label map of the current evaluation record form a corresponding seventh prediction-label pair; and at the end of the round of traversal, the first F1 score corresponding to the first F1 score is obtained according to all the sixth prediction-label pairs obtained in the round of traversal; and the second F1 score corresponding to the second F1 score is obtained according to all the seventh prediction-label pairs obtained in the round of traversal; Step 39, whether the first F1 score and the second F1 score both meet the first F1 score range and the second F1 score range corresponding to each other is identified; if not, it returns to step 31 for continuous training; if yes, it is confirmed that the model training of the first segmentation model is ended.

[0046] Here, the first F1 score range and the second F1 score range are two pre-set numerical ranges.

[0047] Step 4, after the first segmentation model training is completed, a model data set of the second segmentation model is constructed based on the first data set and the first segmentation model, and is recorded as a second data set; The second data set includes a plurality of second data records. The second data record includes a first training image, a first training semantic map, and a third label map. The first training semantic map has a graph shape of D2xH0xW0. The first training semantic map is a second semantic map. Two semantic sub-maps of the first training semantic map are recorded as a gastric antrum training sub-map and a gastroesophageal junction training sub-map. The third label map has a graph shape of D4xH0xW0. The third label map is a third semantic map labeled with a semantic feature label by an artificial labeling method or other machine labeling method. Specifically, each first data record of the first data set is taken as a corresponding current data record, and the first training image of the current data record is taken as a current gastric CT image to be input into the first segmentation model for processing to obtain a corresponding first semantic map and a second semantic map. The second semantic map obtained this time is taken as a corresponding first training semantic map. A corresponding third label map is labeled for the first training image of the current data record in the data format of the third semantic map by an artificial labeling method or other machine labeling method. A corresponding second data record is composed of the first training image, the first training semantic map, and the third label map of the current data record. All the obtained second data records form a corresponding second data set.

[0048] Step 5, training the second segmentation model based on the second data set; Specifically, step 51, the second data set is randomly divided into two sub-data sets based on a pre-set second segmentation ratio, which are recorded as a second training set and a second evaluation set. The second segmentation ratio is a pre-set ratio parameter, for example, 8:2. The second training set and the second evaluation set are both composed of a plurality of second data records. The total number of records of the second training set and the second evaluation set satisfies the second segmentation ratio. Step 52, the first second data record of the second training set is taken as a corresponding current training record. Step 53, the first training image and the first training semantic map of the current training record are taken as a current gastric CT image and a second semantic map to be input into the second segmentation model for processing, and the third semantic map obtained this time is recorded as a corresponding prediction map. The third label map of the current training record is recorded as a corresponding label map. The prediction map and the label map form a corresponding eighth prediction-label loss pair. , ); Step 54, the eighth prediction-label loss pair ( , ) is brought into the preset second model loss function L M2 to obtain the corresponding second loss value; Here, the second model loss function L M2 of the embodiment of the application is: ; Step 55, whether the second loss value meets the preset second loss value range is identified; if the second loss value meets the second loss value range, whether the current training record is the last second data record of the second training set is identified, if yes, step 56 is turned to, if not, the next second data record of the second training set is extracted as a new current training record and step 53 is returned; if the second loss value does not meet the second loss value range, a round of parameter modulation is performed on the second segmentation model based on the preset second model optimizer in the direction of making the second model loss function L M2 reach the minimum value, and step 53 is returned at the end of the round of parameter modulation; Here, the second loss value range is a pre-set numerical range; the second model optimizer includes an Adam optimizer and an SGD optimizer; Step 56, all second data records of the second evaluation set are traversed in a round; and in the round of traversal, the second data record currently traversed is taken as a corresponding current evaluation record; the first training image and the first training semantic graph of the current evaluation record are taken as the current gastric CT image and the second semantic graph, input into the second segmentation model for processing, and the third semantic graph obtained by the processing is taken as a corresponding third prediction graph; and the third label graph of the current evaluation record is taken as a corresponding third prediction graph; and a corresponding ninth prediction-label pair is formed by the third prediction graph and the third label graph of the current evaluation record; and at the end of the round of traversal, the third F1 score is obtained according to all ninth prediction-label pairs obtained in the round of traversal; Step 57, whether the third F1 score meets the preset third F1 score range is identified; if not, step 51 is returned to continue training; if yes, the model training of the second segmentation model is confirmed to be ended.

[0049] Here, the third F1 score range is a pre-set numerical range.

[0050] Step 6, after the second segmentation model training is ended, the first and second segmentation models are used to predict the lesion area of the gastric antrum and the gastroesophageal part of the gastric CT image input by a user and the prediction result is fed back to the current user. Specifically comprising: inputting the user inputted stomach CT image into the first segmentation model for processing to obtain corresponding first and second semantic graphs; inputting the current stomach CT image and the second semantic graph into the second segmentation model for processing to obtain a corresponding third semantic graph; marking the lesion area of the stomach CT image based on the third semantic graph; and feeding back the marked stomach CT image to the current user as the current prediction result.

[0051] Figure 6 A module structure diagram of a lesion area prediction device for the antrum and the gastroesophageal part of the stomach is provided for the second embodiment of the present application. The device is a terminal device or a server for implementing the method embodiments, or a device capable of enabling the terminal device or the server to implement the method embodiments, such as a device or a chip system of the terminal device or the server. As shown in the figure, the device comprises a model construction module 201, a first data preparation module 202, a first model training module 203, a second data preparation module 204, a second model training module 205, and a model application module 206. Figure 6

[0052] The model construction module 201 is configured to construct the first and second segmentation models. The first segmentation model is configured to perform semantic segmentation on the whole stomach region and the antrum and gastroesophageal part region of the model inputted stomach CT image to obtain corresponding first and second semantic graphs. The second semantic graph comprises an antrum semantic subgraph and a gastroesophageal part semantic subgraph. The second segmentation model is configured to perform feature fusion on the model inputted stomach CT image and the second semantic graph, and perform semantic segmentation on the lesion area of the antrum and the gastroesophageal part according to the fused features to obtain a corresponding third semantic graph.

[0053] The first data preparation module 202 is configured to construct a model data set of the first segmentation model by performing big data acquisition on the CT images of stomach cancer patients, which is referred to as a first data set.

[0054] The first model training module 203 is configured to train the first segmentation model based on the first data set. During the training, the prediction and label loss of the first semantic graph is taken as a first loss, the prediction and label loss of the second semantic graph is taken as a second loss, the prediction and label loss of the difference set graph of the first and second semantic graphs is taken as a third loss, the prediction and label loss of the two intersection graphs corresponding to the two sub semantic graphs of the first and second semantic graphs is taken as a fourth and fifth loss, and a corresponding model training loss function is constructed based on the first, second, third, fourth and fifth losses.

[0055] The second data preparation module 204 is configured to construct a model data set of the second segmentation model based on the first data set and the first segmentation model after the training of the first segmentation model is completed, which is referred to as a second data set.

[0056] ​The second model training module 205 trains the second segmentation model based on the second data set.

[0057] The model application module 206 is configured to, after the training of the second segmentation model is completed, perform lesion area prediction of the antrum and the gastroesophageal part based on the first and second segmentation models on the CT image of the stomach input by the user and feed back the prediction result to the current user.

[0058] The device for predicting the lesion area of the antrum and the gastroesophageal part provided by the embodiment of the present application can execute the method steps in the method embodiment, and has similar implementation principles and technical effects, which will not be described here again.

[0059] It should be noted that the division of each module of the above device is only a logical functional division, and all or part of the modules can be integrated into one physical entity, or can be physically separated. These modules can all be implemented in the form of software called by a processing element; all can be implemented in the form of hardware; or part of the modules can be implemented in the form of software called by a processing element, and part of the modules can be implemented in the form of hardware. For example, the model construction module can be a separately established processing element, or can be integrated in a chip of the above device, in addition, the model construction module can also be stored in the form of program code in the memory of the above device, and the functions of the above determination module can be called and executed by a processing element of the above device. The implementation of other modules is similar. In addition, all or part of the modules can be integrated together, or can be independently implemented. The processing element described herein can be an integrated circuit having a signal processing capability. In the implementation process, each step of the above method or each module can be completed by the integrated logic circuit of hardware or the instruction of software in the processing element.

[0060] For example, the above modules can be one or more integrated circuits configured to implement the above method, such as one or more application specific integrated circuits (ASICs), or one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs), etc. For another example, when a certain module above is implemented in the form of scheduling program code by a processing element, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, the modules can be integrated together to implement in the form of system on a chip (SOC).

[0061] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the foregoing method embodiments are generated. The computer described above can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The aforementioned computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the aforementioned computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, Bluetooth, microwave, etc.) means. The aforementioned computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The aforementioned available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks (SSDs)).

[0062] Figure 7 This is a schematic diagram of an electronic device provided in Embodiment 3 of the present invention. This electronic device can be a terminal device or server implementing the methods of the aforementioned embodiments, or it can be a terminal device or server connected to the aforementioned terminal device or server implementing the methods of the aforementioned embodiments. Figure 7 As shown, the electronic device may include: a processor 301 (e.g., CPU), a memory 302, and a transceiver 303; the transceiver 303 is coupled to the processor 301, and the processor 301 controls the transmission and reception operations of the transceiver 303. The memory 302 may store various instructions for performing various processing functions and implementing the processing steps described in the foregoing embodiments. Preferably, the electronic device involved in the embodiments of the present invention further includes: a power supply 304, a system bus 305, and a communication port 306. The system bus 305 is used to realize communication connections between components. The communication port 306 is used for communication between the electronic device and other peripherals.

[0063] exist Figure 7The system bus 305 mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The system bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or only one type of bus. The communication interface is used to realize the communication between the database access device and other devices (such as a client, a read-write library and a read-only library). The memory can contain a Random Access Memory (RAM) and can also include a Non-Volatile Memory, such as at least one disk memory.

[0064] The processor mentioned above can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), a Graphics Processing Unit (GPU), etc.; can also be a Digital Signal Processor (DSP), an Application-Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0065] It should be noted that the embodiments of the present application also provide a computer readable storage medium, which stores instructions, and when the instructions run on a computer, the computer executes the method and processing procedure provided in the above embodiments.

[0066] The embodiments of the present application also provide a chip for running instructions, which is used to execute the processing steps described in the foregoing method embodiments.

[0067] The embodiment of the present application provides a method and device for predicting a lesion area of a gastric antrum and a gastroesophageal part, an electronic device and a computer readable storage medium; from the above content, it can be known that two segmentation models, a first segmentation model and a second segmentation model, are designed to improve the prediction accuracy of the lesion area of the gastric antrum and the gastroesophageal part; wherein the first segmentation model performs semantic segmentation on a whole stomach region and a gastric antrum + gastroesophageal part region of a stomach CT image input by the model to output a corresponding first semantic map (i.e., a conventional whole stomach semantic map) and a second semantic map (i.e., a local gastric antrum and gastroesophageal part semantic map), and the second semantic map can be further divided into two local semantic sub-maps: a gastric antrum part and a gastroesophageal part semantic sub-map; the second segmentation model adds anatomical structure prior information of the second semantic map to the stomach CT image by a feature fusion manner and performs semantic segmentation on the lesion area of the gastric antrum and the gastroesophageal part according to the fused features to obtain a third semantic map (i.e., a lesion semantic map). When training the first segmentation model, three types of constraint losses are added to the two types of conventional losses to improve the prediction accuracy of the model, so that the overall loss function (the first model loss function) is composed of five types of losses: 1) a first loss: a prediction and label loss of the first semantic map, the conventional loss is used to optimize the model parameters for predicting the whole stomach semantic map; 2) a second loss: a prediction and label loss of the second semantic map, the conventional loss is used to optimize the model parameters for predicting the gastric antrum and gastroesophageal part semantic map; 3) a third loss: a prediction and label loss of a difference set map = the first semantic map - the second semantic map (i.e., a stomach body semantic map without the gastric antrum and gastroesophageal part), the loss further improves the prediction accuracy of the first and second semantic maps by constraining the difference set map error; 4) a fourth loss: a prediction and label loss of an intersection map = the first semantic map ∩ the gastric antrum part semantic sub-map (in an ideal state, the intersection map = the gastric antrum part semantic sub-map), the loss further improves the prediction accuracy of the first semantic map and the gastric antrum part semantic sub-map by constraining the intersection map error; 4) a fifth loss: a prediction and label loss of an intersection map = the first semantic map ∩ the gastroesophageal part semantic sub-map (in an ideal state, the intersection map = the gastroesophageal part semantic sub-map), the loss further improves the prediction accuracy of the first semantic map and the gastroesophageal part semantic sub-map by constraining the intersection map error. The first segmentation model of the embodiment of the present application improves the segmentation accuracy of the gastric antrum part and the gastroesophageal part and reduces the segmentation error rate; the second semantic map output by the first segmentation model is input into the second segmentation model as an additional anatomical structure prior feature, which improves the prediction accuracy of the lesion area of the gastric antrum and the gastroesophageal part and reduces the false positive / negative lesion error probability.

[0068] Those skilled in the art should further appreciate that the elements and algorithms described in connection with the examples disclosed herein can be embodied in electronic hardware, computer software, or in combinations of both. To clearly illustrate this interchangeability of hardware and software, various examples have been described herein in terms of their general functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans can implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

[0069] The steps of a method or algorithm described in connection with the examples disclosed herein can be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module can reside in random access memory (RAM), flash memory, read-only memory (ROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0070] The specific implementation described above is further to the purposes, technical solutions, and beneficial effects of the present application. It should be understood that the above description is merely a specific implementation of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for predicting lesion areas in the gastric antrum and gastroesophageal region, characterized in that, The method includes: A first segmentation model and a second segmentation model are constructed. The first segmentation model is used to perform semantic segmentation of the entire gastric region and the gastric antrum and gastroesophageal region on the gastric CT image input to the model to obtain the corresponding first semantic map and second semantic map. The second semantic map includes a semantic sub-map of the gastric antrum and a semantic sub-map of the gastroesophageal region. The second segmentation model is used to perform feature fusion on the gastric CT image input to the model and the second semantic map, and perform semantic segmentation of the lesion region of the gastric antrum and gastroesophageal region according to the fusion features to obtain the corresponding third semantic map. The model dataset for constructing the first segmentation model by collecting large amounts of CT images of gastric cancer patients is denoted as the first dataset. The first segmentation model is trained based on the first dataset; and during training, the prediction and label loss of the first semantic graph is used as the first loss, the prediction and label loss of the second semantic graph is used as the second loss, the prediction and label loss corresponding to the difference graph of the first and second semantic graphs is used as the third loss, and the prediction and label loss of the two intersection graphs corresponding to the two sub-semantic graphs of the first semantic graph and the second semantic graph are used as the fourth and fifth losses, and the corresponding model training loss function is constructed based on the first, second, third, fourth and fifth losses; After the first segmentation model is trained, the model dataset for constructing the second segmentation model based on the first dataset and the first segmentation model is denoted as the second dataset. The second segmentation model is trained based on the second dataset; After the second segmentation model is trained, the lesion areas in the gastric antrum and gastroesophageal region are predicted based on the first and second segmentation models of the gastric CT image input by the user, and the prediction results are fed back to the current user.

2. The method for predicting lesion areas in the gastric antrum and gastroesophageal region according to claim 1, characterized in that, The input end of the first segmentation model is used to receive the gastric CT image, the output end of the first model is used to output the corresponding first semantic map, and the output end of the second model is used to output the corresponding second semantic map. The shape of the gastric CT image is D0×H0×W0, where D0, H0, and W0 are the preset image feature dimensions, image height, and image width, respectively; the gastric CT image is composed of H0×W0 image pixels, and each image pixel includes D0 image features. The shape of the first semantic map is D1×H0×W0, where D1 is a preset first feature dimension and D1=1; the first semantic map is composed of H0×W0 first pixels; each first pixel includes a first semantic feature; the first semantic feature is a binary feature, including 0 and 1; when the first semantic feature is 0, it indicates that the current pixel is not in the whole stomach region, and when it is 1, it indicates that the current pixel is in the whole stomach region; The shape of the second semantic graph is D2×H0×W0, where D2 is the preset second feature dimension and D2=2; the second semantic graph is composed of H0×W0 second pixels, and each second pixel includes two semantic features, namely the second semantic feature and the third semantic feature. The second and third semantic features are each a binary feature, including 0 and 1; the feature combinations of the second and third semantic features have only three combinations, namely [0,0], [0,1] and [1,0]; when the feature combination of the second and third semantic features is [0,0], it means that the current pixel is neither in the antral region nor in the gastroesophageal region, when it is [1,0], it means that the current pixel is in the antral region, and when it is [0,1], it means that the current pixel is in the gastroesophageal region; H0×W0 of the second semantic features of all the second pixels form the corresponding semantic sub-graph of the antral region, and H0×W0 of the third semantic features form the corresponding semantic sub-graph of the gastroesophageal region; The first segmentation model includes a first U-Net model, a first CNN layer, and a second CNN layer; The input terminal of the first U-Net model is connected to the input terminal of the model, and the output terminal is connected to the input terminals of the first CNN layer and the second CNN layer, respectively; the output terminals of the first CNN layer and the second CNN layer are connected to the output terminal of the first model and the output terminal of the second model, respectively. The first U-Net model is implemented based on the conventional U-Net model structure, but does not include the last 1×1 convolutional layer of the conventional U-Net model; the first U-Net model is used to perform feature extraction processing on the gastric CT image to obtain the corresponding feature tensor H1 and send it to the first CNN layer and the second CNN layer; the shape of the feature tensor H1 is D3×H0×W0, where D3 is a preset third feature dimension, and D3>D2; The first CNN layer is used to perform a convolution operation on the feature tensor H1 using a convolution kernel of shape D3×1×1 to obtain the corresponding first semantic map; The second CNN layer is used to perform convolution operations on the feature tensor H1 using two D3×1×1 convolution kernels to obtain the corresponding semantic subgraphs of the gastric antrum and the gastroesophageal region, which together form the corresponding second semantic graph.

3. The method for predicting lesion areas in the gastric antrum and gastroesophageal region according to claim 2, characterized in that, The first and second model input terminals of the second segmentation model are used to receive the corresponding gastric CT image and the second semantic map, and the model output terminal is used to output the corresponding third semantic map; The shape of the third semantic map is D4×H0×W0, where D4 is a preset fourth feature dimension and D4=1; the third semantic map is composed of H0×W0 third pixels; each third pixel includes a fourth semantic feature; the fourth semantic feature is a binary feature, including 0 and 1; when the fourth semantic feature is 0, it indicates that the current pixel is not in the gastric cancer lesion area, and when it is 1, it indicates that the current pixel is in the gastric cancer lesion area; The second segmentation model includes a feature fusion module, a second U-Net model, and a third CNN layer; The first and second input terminals of the feature fusion module are connected to the first model input terminal and the second model input terminal, respectively, and the output terminal is connected to the input terminal of the second U-Net model; the output terminal of the second U-Net model is connected to the input terminal of the third CNN layer; the output terminal of the third CNN layer is connected to the model output terminal. The feature fusion module is used to perform pixel-level feature fusion processing on the gastric CT image and the second semantic map to obtain a corresponding fused feature map, which is then sent to the second U-Net model. The shape of the fused feature map is D5×H0×W0, where D4 is a preset fifth feature dimension, and D5=D0+D2. The fused feature map is composed of H0×W0 fourth pixels. Each fourth pixel includes a first fusion feature. The first fusion feature is composed of the corresponding D0 image features, the second semantic feature, and the third semantic feature. The second U-Net model is implemented based on the conventional U-Net model structure, but does not include the last 1×1 convolutional layer of the conventional U-Net model; the second U-Net model is used to perform feature extraction processing on the fused feature map to obtain the corresponding feature tensor H2 and send it to the third CNN layer; the shape of the feature tensor H2 is D6×H0×W0, where D6 is the preset sixth feature dimension, and D6>D4; The third CNN layer is used to perform convolution operations on the feature tensor H1 using a convolution kernel of shape D6×1×1 to obtain the corresponding third semantic graph.

4. The method for predicting lesion areas in the gastric antrum and gastroesophageal region according to claim 3, characterized in that, The first dataset includes multiple first data records; each first data record includes a first training image, a first label image, and a second label image. The first training image has a shape of D0×H0×W0; the first training image is a CT image of the stomach of a gastric cancer patient; the first label image has a shape of D1×H0×W0; the first label image is a first semantic image with semantic feature labels completed by manual annotation or other machine annotation methods; the second label image has a shape of D2×H0×W0; the second label image is a second semantic image with semantic feature labels completed by manual annotation or other machine annotation methods; the two semantic sub-images of the second label image are denoted as the gastric antrum label sub-image and the gastroesophageal label sub-image; The second dataset includes multiple second data records; the second data records include the first training image, the first training semantic map, and the third label map; The first training semantic map has a shape of D2×H0×W0; the first training semantic map is a second semantic map; the two semantic sub-maps of the first training semantic map are denoted as the gastric antrum training sub-map and the gastroesophageal training sub-map; the third label map has a shape of D4×H0×W0; the third label map is a third semantic map whose semantic feature labels are completed by manual annotation or other machine annotation methods.

5. The method for predicting lesion areas in the gastric antrum and gastroesophageal region according to claim 4, characterized in that, The model dataset used to construct the first segmentation model by collecting large amounts of CT images of gastric cancer patients is denoted as the first dataset, and specifically includes: Step 51: Collect large amounts of data from the CT images of the default scanning plane of the gastric cancer patient through multiple public data acquisition channels to obtain the corresponding raw image set; The publicly available data acquisition channels include publicly available medical image databases and publicly available medical literature / paper databases; the default scanning plane is the coronal plane; and the original image set includes multiple original CT images. Step 52: Using the shape of the gastric CT image as a reference image standard, identify whether the height, width, and feature dimensions of each original CT image in the original image set match the reference image standard; if they match, use the current original CT image as a corresponding preprocessed image; if they do not match, scale or crop the height and width of the current original CT image with reference to the reference image standard to ensure image size matching, and map or align the feature dimensions of the current original CT image with the feature dimensions of the reference image standard one by one to ensure feature dimension matching, and use the processed original CT image as a corresponding preprocessed image; Step 53: Take each of the obtained preprocessed images as the corresponding current image; take the current image as a corresponding first training image; and according to the data format of the first semantic map, label the current image with a corresponding first label map by manual annotation or other machine annotation methods; and according to the data format of the second semantic map, label the current image with a corresponding second label map by manual annotation or other machine annotation methods; and the first training image corresponding to the current image, the first label map, and the second label map form a corresponding first data record; Step 54: The first dataset is composed of all the first data records obtained.

6. The method for predicting lesion areas in the gastric antrum and gastroesophageal region according to claim 4, characterized in that, The step of training the first segmentation model based on the first dataset specifically includes: Step 61: Based on a preset first segmentation ratio, the first dataset is randomly divided into two subsets, denoted as the first training set and the first evaluation set; Wherein, both the first training set and the first evaluation set are composed of multiple first data records; the ratio of the total number of records in the first training set and the first evaluation set satisfies the first segmentation ratio; Step 62: Take the first data record of the first training set as the corresponding current training record; Step 63: Input the first training image recorded in the current training record as the current gastric CT image into the first segmentation model for processing, and use the first semantic map and the second semantic map obtained in this processing as the corresponding prediction maps. and prediction chart ; and the predicted graph The semantic subgraphs of the gastric antrum and the gastroesophageal region are denoted as the corresponding prediction subgraphs. and prediction subgraph And record the first label image and the second label image of the current training record as the corresponding label images. and tag images ; and the label image The semantic subgraphs of the gastric antrum and the gastroesophageal region are denoted as corresponding labeled subgraphs. and tag images ; Step 64, the predicted image and the prediction graph The difference graph is denoted as the corresponding difference graph. ; and the label image and the label image The difference graph is denoted as the corresponding difference graph. ; and the predicted graph and the predicted subgraph The intersection graph is denoted as the corresponding intersection graph. ⊙ represents the Hadamard product; and the predicted graph is... and the predicted subgraph The intersection graph is denoted as the corresponding intersection graph. ; and the label image and the label subgraph The intersection graph is denoted as the corresponding intersection graph. ; and the label image and the label subgraph The intersection graph is denoted as the corresponding intersection graph. ; Step 65, from the predicted graph and the label image Form a corresponding first prediction-label pair ( , ), from the predicted graph and the label image Form a corresponding second prediction-label pair ( , ), from the difference graph and the difference graph Form a corresponding third prediction-label pair ( , ), from the intersection graph and the intersection graph Form a corresponding fourth prediction-label pair ( , ), from the intersection graph and the intersection graph Form a corresponding fifth prediction-label pair ( , ); Step 66: The prediction and label loss of the first semantic graph is used as the first loss; the prediction and label loss of the second semantic graph is used as the second loss; the prediction and label loss corresponding to the difference graph of the first and second semantic graphs is used as the third loss; and the prediction and label loss of the two intersection graphs corresponding to the two sub-semantic graphs of the first and second semantic graphs are used as the fourth and fifth losses. Based on the first, second, third, fourth, and fifth losses, a corresponding first model loss function L is constructed. M1 The first, second, third, fourth, and fifth prediction-label pairs are then input into the first model's loss function L. M1 The corresponding first loss value is obtained through calculation; Wherein, the first model loss function L M1 for: , ， ， ， ， ； L1, L2, L3, L4, and L5 are the corresponding first, second, third, fourth, and fifth losses; the first loss is used to optimize the parameters of the first U-Net model and the first CNN layer; the second loss is used to optimize the parameters of the first U-Net model and the second CNN layer; the third, fourth, and fifth losses are all used to optimize the first U-Net model and to mutually constrain the first CNN layer and the second CNN layer. w1, w2, w3, w4, and w5 are the corresponding first, second, third, fourth, and fifth loss weights; L CE () represents the cross-entropy loss function; Step 67: Identify whether the first loss value meets the preset first loss value range; if the first loss value meets the first loss value range, identify whether the current training record is the last first data record of the first training set; if so, proceed to step 68; otherwise, extract the next first data record of the first training set as the new current training record and return to step 63; if the first loss value does not meet the first loss value range, optimize the first model based on the preset first model optimizer to optimize the first loss L1, second loss L2, third loss L3, fourth loss L4, fifth loss L5 and the first model loss function L M1 Perform a round of parameter modulation on the first segmentation model in the direction where all values ​​reach the minimum, and return to step 63 when the round of parameter modulation ends; The first model optimizer includes the Adam optimizer and the SGD optimizer. Step 68: Perform a traversal of all the first data records in the first evaluation set; during this traversal, the currently traversed first data record is used as the corresponding current evaluation record; the first training image of the current evaluation record is used as the current gastric CT image and input into the first segmentation model for processing; the first semantic map and the second semantic map obtained in this processing are used as the corresponding first prediction map and second prediction map; a corresponding sixth prediction-label pair is formed by the first prediction map and the first label map of the current evaluation record, and a corresponding seventh prediction-label pair is formed by the second prediction map and the second label map of the current evaluation record; at the end of this traversal, an F1 score is evaluated based on all the sixth prediction-label pairs obtained in this traversal to obtain the corresponding first F1 score; and an F1 score is evaluated based on all the seventh prediction-label pairs obtained to obtain the corresponding second F1 score. Step 69: Identify whether the first F1 score and the second F1 score both satisfy their respective first F1 score range and second F1 score range; if not, return to step 61 to continue training; if yes, confirm that the training of the first segmentation model has ended.

7. The method for predicting lesion areas in the gastric antrum and gastroesophageal region according to claim 4, characterized in that, The model dataset for constructing the second segmentation model based on the first dataset and the first segmentation model is denoted as the second dataset, and specifically includes: Each of the first data records in the first dataset is taken as the corresponding current data record; the first training image of the current data record is taken as the current gastric CT image and input into the first segmentation model for processing to obtain the corresponding first semantic map and second semantic map; the second semantic map obtained this time is taken as a corresponding first training semantic map; and according to the data format of the third semantic map, a corresponding third label map is assigned to the first training image of the current data record by manual annotation or other machine annotation methods; and the first training image, the first training semantic map and the third label map corresponding to the current data record are combined to form a corresponding second data record; and all the obtained second data records are combined to form the corresponding second dataset.

8. The method for predicting lesion areas in the gastric antrum and gastroesophageal region according to claim 4, characterized in that, The training of the second segmentation model based on the second dataset specifically includes: Step 81: Based on the preset second segmentation ratio, the second dataset is randomly divided into two subsets, denoted as the second training set and the second evaluation set; The second training set and the second evaluation set are each composed of multiple second data records; the ratio of the total number of records in the second training set and the second evaluation set satisfies the second segmentation ratio. Step 82: Take the first second data record of the second training set as the corresponding current training record; Step 83: Input the first training image and the first training semantic map recorded in the current training record as the current gastric CT image and the second semantic map into the second segmentation model for processing, and record the third semantic map obtained in this processing as the corresponding prediction map. And record the third label image of the current training record as the corresponding label image. ; and from the predicted graph and the label image Form the corresponding eighth prediction-label loss pair ( , ); Step 84, the eighth prediction-label loss pair ( , Substitute the preset second model loss function L M2 The corresponding second loss value is obtained through calculation; Wherein, the second model loss function L M2 for: ； L CE () represents the cross-entropy loss function; Step 85: Identify whether the second loss value meets the preset second loss value range; if the second loss value meets the second loss value range, identify whether the current training record is the last second data record of the second training set; if so, proceed to step 86; otherwise, extract the next second data record of the second training set as the new current training record and return to step 83; if the second loss value does not meet the second loss value range, based on the preset second model optimizer, move towards making the second model loss function L... M2 The second segmentation model is subjected to one round of parameter modulation in the direction where all values ​​reach the minimum, and the process returns to step 83 when the parameter modulation ends. The second model optimizer includes the Adam optimizer and the SGD optimizer; Step 86: Perform a traversal of all the second data records in the second evaluation set; during this traversal, the currently traversed second data record is taken as the corresponding current evaluation record; the first training image and the first training semantic map of the current evaluation record are used as the current gastric CT image and the second semantic map, and input into the second segmentation model for processing; the third semantic map obtained in this processing is recorded as the corresponding third prediction map; the third label map of the current evaluation record is recorded as the corresponding third prediction map; and the third prediction map and the third label map of the current evaluation record form a corresponding ninth prediction-label pair; at the end of this traversal, perform F1 score evaluation based on all the ninth prediction-label pairs obtained in this traversal to obtain the corresponding third F1 score; Step 87: Identify whether the third F1 score meets the preset third F1 score range; if not, return to step 81 to continue training; if yes, confirm that the training of the second segmentation model has ended.

9. The method for predicting lesion areas in the gastric antrum and gastroesophageal region according to claim 1, characterized in that, The process of predicting lesion areas in the gastric antrum and gastroesophageal region based on the user-input gastric CT image using the first and second segmentation models, and then feeding the prediction results back to the current user, specifically includes: The user-inputted gastric CT image is input into the first segmentation model for processing to obtain the corresponding first semantic map and second semantic map; the current gastric CT image and the second semantic map are input into the second segmentation model for processing to obtain the corresponding third semantic map; the gastric CT image is marked with lesion areas based on the third semantic map; and the marked gastric CT image is fed back to the current user as the current prediction result.

10. An apparatus for performing the method for predicting lesion areas in the gastric antrum and gastroesophageal region according to any one of claims 1-9, characterized in that, The device includes: a model building module, a first data preparation module, a first model training module, a second data preparation module, a second model training module, and a model application module; The model building module is used to build a first segmentation model and a second segmentation model. The first segmentation model is used to perform semantic segmentation of the entire gastric region and the gastric antrum and gastroesophageal region on the gastric CT image input to the model to obtain the corresponding first semantic map and second semantic map. The second semantic map includes a semantic sub-map of the gastric antrum and a semantic sub-map of the gastroesophageal region. The second segmentation model is used to perform feature fusion on the gastric CT image input to the model and the second semantic map, and perform semantic segmentation of the lesion region of the gastric antrum and gastroesophageal region according to the fusion features to obtain the corresponding third semantic map. The first data preparation module is used to construct the model dataset of the first segmentation model by collecting large amounts of data from CT images of gastric cancer patients. This dataset is denoted as the first dataset. The first model training module trains the first segmentation model based on the first dataset; and during training, the prediction and label loss of the first semantic graph is used as the first loss, the prediction and label loss of the second semantic graph is used as the second loss, the prediction and label loss corresponding to the difference graph of the first and second semantic graphs is used as the third loss, and the prediction and label loss of the two intersection graphs corresponding to the two sub-semantic graphs of the first semantic graph and the second semantic graph are used as the fourth and fifth losses, and the corresponding model training loss function is constructed based on the first, second, third, fourth and fifth losses; The second data preparation module is used to construct a model dataset for the second segmentation model based on the first dataset and the first segmentation model after the first segmentation model has been trained. This dataset is referred to as the second dataset. The second model training module trains the second segmentation model based on the second dataset; The model application module is used to predict the lesion areas of the gastric antrum and gastroesophageal region based on the first and second segmentation models after the training of the second segmentation model is completed, and to feed back the prediction results to the current user.

11. An electronic device, characterized in that, include: Memory, processor, and transceiver; The processor is configured to be coupled to the memory, read and execute instructions in the memory to implement the method according to any one of claims 1-9; The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a computer, cause the computer to perform the method according to any one of claims 1-9.

Images