Postoperative pathology assessment method, survival prediction method and system for esophageal cell carcinoma

Abstract

The invention discloses a postoperative pathology assessment method, survival prediction method and system for esophageal cell carcinoma, and the method comprises the steps: obtaining and preprocessing WSI image data, obtaining cut small images of the WSI image data in a low-magnification view angle, a medium-magnification view angle and a high-magnification view angle, and sequentially inputting the cut small images into a selective spatial state model through feature extraction; updating the diagnosis states through a long and short term memory network, and adjusting the contribution degree of each diagnosis state through a selective memory weight; fusing all diagnosis states, and outputting the probability of tissue types through a full connection layer and a Softmax function; calculating the percentage of the residual living tumor, and obtaining a pathological assessment result according to the percentage of the residual living tumor. According to the method, the diagnosis process of doctors can be simulated, the workload of pathologists can be reduced, the evaluation consistency and universality are improved, and the survival rate can be predicted and clinical treatment decision making can be assisted through the constructed survival prediction model.

Description

Technical Field

[0001] The present invention relates to the technical field of pathological evaluation and survival prediction, and in particular to a postoperative pathological evaluation method, a survival prediction method and a system for esophageal cell carcinoma. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Data show that esophageal cancer is one of the malignant tumors with high morbidity and mortality. Esophageal squamous cell carcinoma (ESCC) is the main histological subtype of esophageal cancer, and ESCC has a poor prognosis. Patients with locally advanced esophageal cancer usually choose preoperative chemotherapy, but the efficacy of this method has reached a plateau. The emergence of immunotherapy has brought new breakthroughs in the treatment of esophageal cancer, and preoperative chemotherapy combined with immunotherapy has been developed.

[0004] However, there are currently many difficulties in evaluating the pathological response after preoperative chemotherapy combined with immunotherapy for ESCC. In order to obtain a more accurate description in the prior art, pathologists need to perform more diagnostic readings, which increases the workload of pathological evaluation; due to the differences in grading systems, diagnostic grading, and diagnostic descriptions in the prior art, the evaluation consistency of the prior art is poor. These inconsistencies may cause confusion and misjudgment in the communication and learning between physicians, and even affect the diagnosis and treatment decision-making; and because the evaluation of pathological response requires experienced pathological diagnostic experts, such talents are very scarce even in regional pathological diagnosis centers, and the training cycle is long, and it is difficult to complete personnel expansion in a short period of time, which makes the evaluation of pathological response in the prior art difficult to popularize. In addition, the pathological evaluation model in the prior art cannot simulate the doctor's diagnostic process, resulting in low accuracy of the pathological evaluation results and poor evaluation effect.

[0005] At the same time, the current ESCC preoperative chemotherapy combined with immunotherapy survival prediction based on pathological images also has certain defects. Most of the existing technologies are based on untreated tissue samples, extracting pathological features and making survival predictions, which are not suitable for the model of preoperative neoadjuvant chemotherapy combined with immunotherapy for esophageal cancer. In addition, there is limited evidence on the impact of the extracted histological features on survival prognosis and lack of strong theoretical support. Summary of the invention

[0006] In order to solve the above problems, the present invention proposes a postoperative pathological evaluation method, a survival prediction method and a system for esophageal cell carcinoma. The present invention can simulate the doctor's diagnosis process, reduce the workload of pathologists, improve the consistency and popularity of the evaluation, and through the constructed survival prediction model, the survival rate can be predicted to assist in clinical treatment decision-making.

[0007] In order to achieve the above object, the present invention adopts the following technical solution: In a first aspect, the present invention provides a method for postoperative pathological evaluation of esophageal cell carcinoma, comprising: Acquire the patient's HE slice data, scan and generate WSI image data, and preprocess the WSI image data to obtain cropped thumbnails of the WSI image data at low, medium and high magnification viewing angles; Feature extraction is performed on the cropped images at low, medium and high magnification viewing angles to obtain image features at different viewing angles, namely overall features, local features and detail features. The image features at different viewing angles are sequentially input into the selective spatial state model. The image features at each viewing angle are regarded as the input at the current moment, and the corresponding diagnostic state is obtained, wherein the diagnostic state is updated through the long short-term memory network, and the contribution of each diagnostic state is adjusted through the selective memory weight; all diagnostic states are fused, and the probability of tissue type is output through the fully connected layer and the Softmax function; According to the probability of the tissue type, the diagnosis result of the cropped image is obtained, and the residual viable tumor percentage is calculated according to the diagnosis result. According to the residual viable tumor percentage, the pathological evaluation result is obtained.

[0008] According to a further technical solution, after the WSI image data is generated, the tumor bed area and the residual tumor area are outlined, and then the WSI image data is preprocessed.

[0009] A further technical solution is a method for preprocessing the WSI image data: first, a thumbnail image of a preset magnification is obtained based on the read WSI image data, the thumbnail image is converted into a grayscale image, and then the grayscale image is divided into a tissue area and a background area; then, the WSI image data is cropped into different types of thumbnails according to the tumor bed area, the residual tumor area and the tissue area, including: a tumor bed thumbnail, a residual tumor thumbnail and other tissue thumbnails, respectively. After obtaining different types of thumbnails, the coordinates of each thumbnail are mapped to images of different magnifications, and cropped thumbnails of the WSI image data at low, medium and high magnification viewing angles are obtained from the mapped areas; finally, color correction is performed on all thumbnails.

[0010] In a further technical solution, the updated diagnosis status includes the image features input at the current moment and the diagnosis status at the previous moment. The specific updating method is: ;in, Indicates the updated diagnostic status; Indicates the diagnostic status at the previous moment; Represents the image features input at the current moment.

[0011] In a further technical solution, the selective memory weight is calculated by a Sigmoid activation function, and the formula is: ;in, represents the selective memory weight, ( ) represents the Sigmoid activation function, and Represents the learning parameters, controlling the degree of memory enhancement for each perspective; All diagnostic states are fused into global features, and the fused global features are expressed as: ;in, Represents global features, Represents a learnable coefficient that controls the image features of each view Contribution to global features.

[0012] A further technical solution is to obtain the diagnostic result of the cropped image based on the probability of the tissue type, determine the tumor bed area and the residual viable tumor area, and calculate the residual viable tumor percentage as follows: ;in, is the percentage of residual viable tumor, Sb is the area of ​​the tumor bed in the HE section, St It is the area of ​​residual viable tumor in HE sections; according to the percentage of residual viable tumor, the pathological evaluation results are divided into no tumor cells, significant pathological remission, partial pathological remission and no pathological remission.

[0013] In a further technical solution, the loss function of the selective spatial state model is a cross entropy function, and the specific calculation method is: ;in, is the number of label types, y ( k ) is a one-hot data label, p ( k ) is the probability of tissue type output by the pathology assessment model.

[0014] In a second aspect, the present invention provides a postoperative pathological evaluation system for esophageal cell carcinoma, comprising: The data acquisition and preprocessing module is configured to: acquire HE slice data of the patient, scan and generate WSI image data, and preprocess the WSI image data to obtain cropped thumbnails of the WSI image data at low, medium and high magnification viewing angles after the preprocessing; The feature extraction module is configured to: extract features of the cropped small images at low-magnification viewing angle, medium-magnification viewing angle and high-magnification viewing angle respectively to obtain image features at different viewing angles, namely, overall features, local features and detail features; The data processing module is configured to: input the image features under different viewing angles into the selective spatial state model in sequence, the image features of each viewing angle are regarded as the input at the current moment, and the corresponding diagnostic state is obtained, wherein the diagnostic state is updated through the long short-term memory network, and the contribution of each diagnostic state is adjusted through the selective memory weight; all diagnostic states are merged, and the probability of tissue type is output through the fully connected layer and the Softmax function; The pathological assessment module is configured to obtain a diagnosis result of the cropped image according to the probability of the tissue type, calculate the residual viable tumor percentage according to the diagnosis result, and obtain a pathological assessment result according to the residual viable tumor percentage.

[0015] In a third aspect, the present invention provides a method for predicting postoperative survival of esophageal cell carcinoma, based on the postoperative pathological evaluation method of esophageal cell carcinoma described in any one of the first aspects, comprising: Obtaining patient survival characteristics, and dividing the patients into a high-risk group and a low-risk group according to the median of the patient survival characteristics; The Log-rank test was used to calculate the difference in survival curves between the high-risk group and the low-risk group, and the survival characteristics were preliminarily screened. The LASSO Cox regression model was used to further screen the survival characteristics. The LASSO Cox regression model was optimized using the minimum ten-fold cross-validation MSE to obtain survival characteristics with non-zero coefficients and input them into the multivariate Cox proportional hazard model to output the survival prediction results.

[0016] In a fourth aspect, the present invention provides a postoperative survival prediction system for esophageal cell carcinoma, comprising: The survival feature acquisition module is configured to: acquire the patient's survival feature, and divide the patient into a high-risk group and a low-risk group according to the median of the patient's survival feature; The survival prediction module is configured as follows: using the Log-rank test to calculate the difference in survival curves between the high-risk group and the low-risk group of patients, and preliminarily screening the survival characteristics; using the LASSO Cox regression model to further screen the survival characteristics; using the minimum ten-fold cross-validation MSE to optimize the LASSO Cox regression model, obtain the survival characteristics with non-zero coefficients and input them into the multivariate Cox proportional hazard model, and output the survival prediction results.

[0017] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention constructs a Mamba model based on multi-perspective memory enhancement, the core of which is the Selective Space State Model (SSM), which can simulate the selective memory process of pathologists in the diagnosis process and improve the accuracy of evaluation: the present invention simulates the doctor's diagnosis process, integrates the image features of different perspectives for comprehensive diagnosis, and selectively enhances the memory of image features of different magnifications. The selective space state model is used to perform deep learning on pathological sections, which can accurately identify pathological image features and the correlation between features, and is used to obtain the probabilities of different pathological tissue types, thereby accurately evaluating pathological reactions, greatly improving the accuracy of pathological evaluation, and increasing the diagnostic accuracy to more than 90%.

[0018] 2. The present invention uses deep learning by machine, so pathologists do not need to perform too much diagnostic film reading, which significantly reduces the workload of pathological diagnosis; the present invention avoids the problem of poor evaluation consistency caused by different grading systems, diagnostic grading and diagnostic descriptions, reduces confusion and misjudgment in communication and learning between doctors, and provides a more reliable basis for diagnosis and treatment decision-making; at the same time, grassroots hospitals can use the pathological evaluation model provided by the present invention to solve the problem of difficulty in pathological evaluation caused by the shortage of professional talents, and improve the popularity of pathological evaluation after preoperative chemotherapy combined with immunotherapy for ESCC.

[0019] 3. The present invention constructs a postoperative prognosis survival prediction model based on the obtained pathological evaluation results and combined with clinical characteristic factors, which can screen out the key factors affecting survival, thereby formulating personalized treatment plans for patients in a targeted manner, improving the accuracy and effectiveness of postoperative treatment, and also improving the accuracy of survival prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0021] Figure 1 It is a structural diagram of the pathological assessment model of the present invention. DETAILED DESCRIPTION

[0022] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0023] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those of ordinary skill in the art to which the present invention belongs. In the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0024] Embodiment 1 This embodiment provides a postoperative pathological evaluation method for esophageal cell carcinoma, which specifically includes the following steps: S1. Acquire HE slice data of a patient, scan and generate WSI image data, and preprocess the WSI image data to obtain cropped thumbnails of the WSI image data at low, medium and high magnification viewing angles; S2, extracting features from the cropped images at low, medium and high magnification viewing angles to obtain image features at different viewing angles, namely, overall features, local features and detail features; S3. Input the image features at different viewing angles into the selective spatial state model in sequence, and the image features at each viewing angle are regarded as the input at the current moment, and the corresponding diagnostic state is obtained, wherein the diagnostic state is updated through the long short-term memory network, and the contribution of each diagnostic state is adjusted through the selective memory weight; all diagnostic states are fused, and the probability of tissue type is output through the fully connected layer and the Softmax function; S4. Obtain the diagnosis result of the cropped image according to the probability of the tissue type, calculate the residual viable tumor percentage according to the diagnosis result, and obtain the pathological evaluation result according to the residual viable tumor percentage.

[0025] Among them, in step S1, for the enrolled patients, the HE slice data were scanned in multiple layers at a resolution of 0.25 μm / pixel (40 times magnification) using a Pannoramic MIDI slice scanner (3DHISTECH Ltd., Budapest, Hungary) to generate WSI image data for subsequent analysis. After the WSI image data were generated, the pathologists outlined the tumor bed area and the residual tumor area as the gold standard (i.e., classification label) in the study, and then the WSI image data were preprocessed.

[0026] Before building a deep learning model, you first need to preprocess the WSI image data to convert it into multiple small images that can be used for model analysis. The specific method for preprocessing the WSI image data is as follows: First, the WSI image data is read from the OpenSlide library, and a thumbnail image with a preset magnification is obtained. In this embodiment, the preset magnification is 2.5 times (4 / pixel), and then use OpenCV's cv2.COLOR_RGB2GRAY to convert the thumbnail image to a grayscale image, and then use cv2.THRESH_BINARY_INV to segment the grayscale image into tissue areas (less than the threshold) and background areas (greater than the threshold) at a threshold of 220; then subdivide the tissue area on the WSI image data into non-overlapping small images. Specifically: at a magnification of 2.5 times, based on the tumor bed area, residual tumor area, and tissue area on the WSI image data outlined by pathologists, the WSI image data is cropped to a size of 224 224 Three different types of small images, including: tumor bed small image, residual tumor small image and other tissue small image. After obtaining different types of small images, if a small image contains more than one tissue type, the tissue type with an area greater than 50% is used as the tissue type of its small image. Subsequently, the coordinates of each small image are mapped to images of different magnifications (5x, 10x, 20x, 40x), and 4, 16, 64 and 256 images of size 224 are cropped from the mapped area. 224 Cropped images of different magnification angles. Among them, 2.5 times is a low magnification angle, 5 times and 10 times are medium magnification angles, and 20 times and 40 times are high magnification angles.

[0027] After obtaining small images of different types, the coordinates of each small image are mapped to images of different magnifications, and cropped small images of different magnification viewing angles are obtained from the mapped areas. Finally, color correction is performed on all small images. Specifically, in this embodiment, the Vahadane method of the StainTools tool is used to perform color correction on all small images, and its staining density map is combined with the color distribution of the target image selected by the pathologist, so as to unify the color distribution of all small images and reduce the impact of staining differences on subsequent analysis. At the same time, the structural information of the image is retained so that the model can focus on the structural differences in the image and improve the tissue identification ability of the model.

[0028] After the above preprocessing, in step S2, feature extraction is performed on the cropped images at low, medium and high magnification perspectives to obtain image features at different perspectives. The cropped images at each perspective are processed by a convolutional neural network (CNN) to extract the overall, local and detailed image features from low to medium and high magnification perspectives. The low magnification perspective (2.5 times) focuses on the overall tissue structure, while the high magnification perspective (40 times) focuses on the detailed features at the cellular level. To this end, different convolutional neural networks are used to extract effective features from images of different magnifications. Specifically: Low magnification (2.5x): Use the pre-trained ResNet50 network to extract overall features, mainly learning the macroscopic structure of the tissue, such as shape, size, distribution, etc.

[0029] Medium magnification (5x, 10x): A deeper ResNet152 network is used to capture local features and focus on detailed differences between tissues, such as tumor boundaries and blood vessels.

[0030] High-magnification perspective (20x, 40x): Use a network with detail capture capabilities (U-Net) to extract detailed features at the cellular level, focusing on microscopic information such as cell morphology and cell spacing.

[0031] Finally, the features extracted from images at different perspectives are F i ,in i They are 2.5, 5, 10, 20 and 40 times magnification respectively.

[0032] In step S3, the present embodiment constructs a Mamba model based on multi-perspective memory enhancement, and its core structure is the selective state space model (SSM). In order to simulate the selective memory process of pathologists during the diagnosis process, the present embodiment uses a parameterized selective SSM to learn the importance of image features of each perspective to the diagnosis. Each perspective (magnification) is regarded as a time point and a time step, and the image features under different perspectives are input into the selective space state model in sequence, that is, the low-magnification image features (such as 2.5 times) are used as the earliest input, and the high-magnification image features (such as 40 times) are used as the latest input. All image features are input into the selective space state model in chronological order. The core of the model is to strengthen the memory of the diagnosis process through the image features of each perspective. The image features of each perspective F i is regarded as the current moment (time point t= i ) input, each time the image feature is input F i The corresponding diagnostic status will be obtained, and the diagnostic status is updated through the Long Short-Term Memory (LSTM) network. i , its image features F i and the diagnostic status at the previous moment s i-1 It is input into the LSTM network for updating. The updated diagnosis state includes the image features input at the current moment and the diagnosis state at the previous moment. The updated diagnosis state si can be expressed by the following formula: ;in, Indicates the updated diagnostic status; Indicates the diagnostic status at the previous moment; Represents the image features input at the current moment.

[0033] In order to simulate the pathologist's memory of different magnification image features during the diagnosis process, this embodiment introduces a selective memory mechanism. To adjust the contribution of each diagnostic status. Memory weight Calculated by Sigmoid activation function, the formula is as follows: ;in, represents the selective memory weight, ( ) represents the Sigmoid activation function, and represents the parameters of learning, controlling the degree of memory reinforcement for each view; in this way, crops from low-magnification views (such as 2.5x) may be given lower weights, while crops from high-magnification views (such as 40x) may be given higher weights, and the model dynamically adjusts the importance of the image features of each view to the final diagnosis.

[0034] The diagnostic status of each perspective is updated and its importance is adjusted through the selective memory mechanism. Then the diagnostic status of all perspectives needs to be fused into a global feature for the final classification decision. Feature fusion is performed by weighted summation. The fused global feature It can be expressed as: ;in, Represents global features, Represents a learnable coefficient that controls the image features of each view Contribution to global features.

[0035] The Mamba model of this embodiment adopts a hardware-aware design. Considering the processing requirements of large-scale pathological images, the model will perform parallel calculations as much as possible during the reasoning process to improve efficiency. During the training process, an efficient GPU cluster is used for parallel computing to accelerate the processing of large-scale image data. At the same time, the architecture of the model is optimized to adapt to hardware resources to ensure that it can still run efficiently under limited computing resources. The model in this embodiment is trained and verified based on the Pytorch 1.9.0 framework on an Ubuntu server equipped with an NVIDIA GeForce RTX 4090 GPU.

[0036] During the training process of this embodiment, the optimization goal is to minimize the classification loss of tissue types. The loss function of the model is the cross entropy (CE) function, and the specific calculation method is: ;in, is the number of tag types, y ( k ) is the data label in one - hot form, p ( k ) is the probability of the tissue type output by the pathological evaluation model.

[0037] During the training process, the Adam optimizer is used, and the formula is: ; Among them, are the parameters of the current model, is the learning rate, is the gradient of the loss function.

[0038] In this embodiment, all patients are assigned to the training set, test set, and validation set in a ratio of 3:1:1. The training set is used to optimize the hyperparameters of the deep learning network; the test set is used to select the model with the best performance; the validation set is used to verify the diagnostic ability of the model. Standard evaluation metrics (accuracy, precision, recall, F1 - score, etc.) are used to verify the performance of the model.

[0039] In step S3, according to the probability of the tissue type, the diagnostic result of the cropped small image is obtained, and the tumor bed area and the residual viable tumor area are determined. The calculation of the residual viable tumor percentage is specifically as follows: ; among them, is the residual viable tumor percentage, Sb is the area of the tumor bed in the HE section, St is the area of the residual viable tumor in the HE section. According to the residual viable tumor percentage, the pathological evaluation results are divided into no tumor cells seen, significant pathological remission (RVT ≤ 10%), partial pathological remission (10% < RVT < 90%), and no pathological remission (RVT ≥ 90%).

[0040] The training and testing of the model are both completed at the small - image level. After the model is trained, each sub - image at 40 - fold magnification is combined with its corresponding four lower - magnification sub - images (20 - fold, 10 - fold, 5 - fold, 2.5 - fold). After feature extraction, an image feature matrix is formed and input into the trained Mamba model for inference to predict its tissue type, so as to obtain the tissue type of each sub - image at 40 - fold magnification on the whole pathological slide. If the number of small images predicted as tumor regions is , and the number of small images predicted as tumor beds is , then, based on the deep - learning model, the is calculated as: ; finally, referring to the Cottrell pathological response evaluation system, the tumor regression grade is determined.

[0041] Embodiment 2 This embodiment discloses a postoperative pathological evaluation system for esophageal cell carcinoma, comprising: The data acquisition and preprocessing module is configured to: acquire HE slice data of the patient, scan and generate WSI image data, and preprocess the WSI image data to obtain cropped thumbnails of the WSI image data at low, medium and high magnification viewing angles after the preprocessing; The feature extraction module is configured to: extract features of the cropped small images at low-magnification viewing angle, medium-magnification viewing angle and high-magnification viewing angle respectively to obtain image features at different viewing angles, namely, overall features, local features and detail features; The data processing module is configured to: input the image features under different viewing angles into the selective spatial state model in sequence, the image features of each viewing angle are regarded as the input at the current moment, and the corresponding diagnostic state is obtained, wherein the diagnostic state is updated through the long short-term memory network, and the contribution of each diagnostic state is adjusted through the selective memory weight; all diagnostic states are merged, and the probability of tissue type is output through the fully connected layer and the Softmax function; The pathological assessment module is configured to obtain a diagnosis result of the cropped image according to the probability of the tissue type, calculate the residual viable tumor percentage according to the diagnosis result, and obtain a pathological assessment result according to the residual viable tumor percentage.

[0042] Embodiment 3 This embodiment discloses a method for predicting postoperative survival of esophageal cell carcinoma, based on the method for pathological evaluation of esophageal cell carcinoma after preoperative chemotherapy combined with immunotherapy as described in any one of the first embodiments, including: First, the patient survival characteristics were obtained, including survival information, pathological evaluation results, gender, age, ECOG score, BMI, lesion location, lesion length, TNM clinical stage before treatment, PD-L1 expression status, TMB status, MSI status, and immune microenvironment status in postoperative specimens.

[0043] Then, according to the median of each of the above survival characteristics, the patients were divided into high-risk group and low-risk group; the difference in survival curves of patients in the high-risk group and low-risk group was calculated using the Log-rank test, and the survival characteristics were preliminarily screened, specifically, the characteristics with p < 0.05 were preliminarily screened; then, the LASSO Cox regression model was used to further screen the survival characteristics; the LASSO Cox regression model was optimized using the minimum ten-fold cross-validation MSE, and the survival characteristics with non-zero coefficients were obtained and input into the multivariate Cox proportional hazard model to output the survival prediction results. The C-index, Kaplan-Meier survival curve and receiver operating characteristic (ROC) curve were used to evaluate the predictive ability of the model.

[0044] Embodiment 4 This embodiment discloses a postoperative survival prediction system for esophageal cell carcinoma, comprising: The survival feature acquisition module is configured to: acquire the patient's survival feature, and divide the patient into a high-risk group and a low-risk group according to the median of the patient's survival feature; The survival prediction module is configured as follows: using the Log-rank test to calculate the difference in survival curves between the high-risk group and the low-risk group of patients, and preliminarily screening the survival characteristics; using the LASSO Cox regression model to further screen the survival characteristics; using the minimum ten-fold cross-validation MSE to optimize the LASSO Cox regression model, obtain the survival characteristics with non-zero coefficients and input them into the multivariate Cox proportional hazard model, and output the survival prediction results.

[0045] It should be noted here that each module in this embodiment corresponds one-to-one to the method in Embodiment 1, and the specific implementation process is the same, which will not be repeated here.

[0046] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

[0047] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A method for postoperative pathological evaluation of esophageal cell carcinoma, characterized in that: include: Acquire the patient's HE slice data, scan and generate WSI image data, and preprocess the WSI image data to obtain cropped thumbnails of the WSI image data at low, medium and high magnification viewing angles; Feature extraction is performed on the cropped images at low, medium and high magnification viewing angles to obtain image features at different viewing angles, namely overall features, local features and detail features. The image features at different viewing angles are sequentially input into the selective spatial state model, and the image features at each viewing angle are regarded as the input at the current moment, and the corresponding diagnostic state is obtained, wherein the diagnostic state is updated through the long short-term memory network, and the contribution of each diagnostic state is adjusted through the selective memory weight; All diagnostic states are integrated and the probability of tissue type is output through the fully connected layer and Softmax function; According to the probability of the tissue type, the diagnosis result of the cropped image is obtained, and the residual viable tumor percentage is calculated according to the diagnosis result. According to the residual viable tumor percentage, the pathological evaluation result is obtained.

2. The postoperative pathological evaluation method for esophageal cell carcinoma according to claim 1, characterized in that: After the WSI image data is generated, the tumor bed area and the residual tumor area are outlined, and then the WSI image data is preprocessed.

3. The postoperative pathological evaluation method for esophageal cell carcinoma according to claim 2, characterized in that: The method for preprocessing the WSI image data is as follows: firstly, a thumbnail image with a preset magnification is obtained according to the read WSI image data, the thumbnail image is converted into a grayscale image, and then the grayscale image is segmented into a tissue area and a background area; Then, the WSI image data is cropped into different types of thumbnails according to the tumor bed area, residual tumor area and tissue area, including: tumor bed thumbnail, residual tumor thumbnail and remaining tissue thumbnail. After obtaining different types of thumbnails, the coordinates of each thumbnail are mapped to images of different magnifications, and the cropped thumbnails of the WSI image data at low, medium and high magnification viewing angles are obtained from the mapped areas; finally, color correction is performed on all thumbnails.

4. The postoperative pathological evaluation method for esophageal cell carcinoma according to claim 1, characterized in that: The updated diagnosis status includes the image features input at the current moment and the diagnosis status at the previous moment. The specific updating method is: ;in, Indicates the updated diagnostic status; Indicates the diagnostic status at the previous moment; Represents the image features input at the current moment.

5. The postoperative pathological evaluation method for esophageal cell carcinoma according to claim 1, characterized in that: The selective memory weight is calculated by the Sigmoid activation function, and the formula is: ;in, represents the selective memory weight, ( ) represents the Sigmoid activation function, and Represents the learning parameters, controlling the degree of memory enhancement for each perspective; All diagnostic states are fused into global features, and the fused global features are expressed as: ;in, Represents global features, Represents a learnable coefficient that controls the image features of each view Contribution to global features.

6. The postoperative pathological evaluation method for esophageal cell carcinoma according to claim 1, characterized in that: According to the probability of tissue type, the diagnostic results of the cropped image are obtained, and the tumor bed area and residual viable tumor area are determined. The percentage of residual viable tumor is calculated as follows: ;in, is the percentage of residual viable tumor, Sb is the area of ​​the tumor bed in the HE section, St It is the area of ​​residual viable tumor in HE sections; according to the percentage of residual viable tumor, the pathological evaluation results are divided into no tumor cells, significant pathological remission, partial pathological remission and no pathological remission.

7. The postoperative pathological evaluation method for esophageal cell carcinoma according to claim 1, characterized in that: The loss function of the selective spatial state model is the cross entropy function, which is specifically calculated as follows: ;in, is the number of label types, y ( k ) is a one-hot data label, p ( k ) is the probability of tissue type output by the pathology assessment model.

8. A postoperative pathological evaluation system for esophageal cell carcinoma, characterized in that: include: The data acquisition and preprocessing module is configured to: acquire HE slice data of the patient, scan and generate WSI image data, and preprocess the WSI image data to obtain cropped thumbnails of the WSI image data at low, medium and high magnification viewing angles after the preprocessing; The feature extraction module is configured to: extract features of the cropped small images at low-magnification viewing angle, medium-magnification viewing angle and high-magnification viewing angle respectively to obtain image features at different viewing angles, namely, overall features, local features and detail features; The data processing module is configured to: input the image features under different viewing angles into the selective spatial state model in sequence, the image features of each viewing angle are regarded as the input at the current moment, and the corresponding diagnostic state is obtained, wherein the diagnostic state is updated through the long short-term memory network, and the contribution of each diagnostic state is adjusted through the selective memory weight; all diagnostic states are merged, and the probability of tissue type is output through the fully connected layer and the Softmax function; The pathological assessment module is configured to obtain a diagnosis result of the cropped image according to the probability of the tissue type, calculate the residual viable tumor percentage according to the diagnosis result, and obtain a pathological assessment result according to the residual viable tumor percentage.

9. A method for predicting postoperative survival of esophageal cell carcinoma, based on the postoperative pathological evaluation method of esophageal cell carcinoma according to any one of claims 1 to 7, characterized in that: include: Obtaining patient survival characteristics, and dividing the patients into a high-risk group and a low-risk group according to the median of the patient survival characteristics; The Log-rank test was used to calculate the difference in survival curves between the high-risk group and the low-risk group, and the survival characteristics were preliminarily screened; The LASSO Cox regression model was used to further screen the survival characteristics. The LASSO Cox regression model was optimized using the minimum ten-fold cross-validation MSE to obtain the survival characteristics with non-zero coefficients and input them into the multivariate Cox proportional hazard model to output the survival prediction results.

10. A postoperative survival prediction system for esophageal cell carcinoma, characterized in that: include: The survival feature acquisition module is configured to: acquire the patient's survival feature, and divide the patient into a high-risk group and a low-risk group according to the median of the patient's survival feature; The survival prediction module is configured to: calculate the difference in survival curves between the high-risk group and the low-risk group using a Log-rank test, and preliminarily screen survival characteristics; The LASSO Cox regression model was used to further screen the survival characteristics. The LASSO Cox regression model was optimized using the minimum ten-fold cross-validation MSE to obtain the survival characteristics with non-zero coefficients and input them into the multivariate Cox proportional hazard model to output the survival prediction results.

Images