Method for predicting lymph node state in rectal cancer area based on nnUnetV2 and radiomics

Abstract

The invention discloses a method for predicting the lymph node state of a rectal cancer area based on nnUnetV2 and radiomics. The method comprises the following steps: collecting data; carrying out VOI sketching on a rectal cancer tumor area; preprocessing the MRI image; training an automatic segmentation model; image omics feature extraction; feature dimension reduction and selection; and constructing a prediction model by adopting logistic regression LR, and predicting the lymph node state. According to the method, automatic segmentation based on deep learning nnU-NetV2 is combined with MRI image omics features, the problems of time and labor consumption and low efficiency of manual marking by a doctor in a traditional image omics method are solved, the image omics features which can better reflect the actual condition of a training sample can be extracted from medical images of different modalities, and the accuracy of the image omics feature extraction is improved. Auxiliary diagnosis is provided for noninvasive prediction of regional lymph node states of rectal cancer patients, accurate prediction of lymph nodes is facilitated, and a more accurate and personalized diagnosis and treatment scheme is provided for colorectal cancer.

Description

Technical Field

[0001] The present invention relates to the fields of precise segmentation technology and machine learning prediction for rectal cancer, and specifically to a method for predicting the status of rectal cancer lymph nodes based on radiomics feature extraction based on automatic segmentation of target areas. The method combines radiomics and deep learning to effectively predict whether rectal cancer has metastatic regional lymph nodes. Background Art

[0002] Colorectal cancer ranks third in global cancer incidence and second in mortality. Lymph node status is one of the important factors affecting the treatment strategy and prognosis of colorectal cancer patients. Because magnetic resonance imaging (MRI) has high soft tissue resolution and is superior to computed tomography (CT) in displaying tumors and surrounding anatomical structures, MRI is considered the preferred imaging examination for colorectal cancer staging. The combined use of high-resolution T2-weighted imaging and conventional diffusion-weighted imaging can achieve an accuracy of over 90% in determining the depth of rectal cancer invasion (T stage). However, for the assessment of regional lymph node metastasis (N stage), its sensitivity and specificity have not reached 80%. Therefore, the accuracy of imaging assessment of lymph node status needs to be further improved.

[0003] Radiomics is an advanced image analysis method that uses computer software to extract a large number of high-dimensional features from medical images at high throughput. This allows for deeper mining, prediction, and analysis of massive amounts of imaging data to assist physicians in making the most accurate clinical decisions. In practice, the lesion regions relied upon by radiomics analysis require manual labeling by professional physicians, which is time-consuming and subject to subjective bias. Consequently, processing images from complex imaging scenarios often results in suboptimal tumor segmentation and poor reproducibility of radiomics feature extraction. Although there are studies (Ma S, Lu H, Jing G, Li Z, Zhang Q, Ma X, Chen F, Shao C, Lu Y, Wang H, Shen F. Deep learning-based clinical-radiomicsnomogram for preoperative prediction of lymph node metastasis in patients with rectal cancer: a two-center study. Front Med (Lausanne). 2023 Dec 1; 10: 1276672. doi: 10.3389 / fmed.2023.1276672. PMID: 38105891; PMCID: PMC10722265.) based on nnUnet (short for no-new-Net, an adaptive medical image segmentation framework developed by the Medical Image Computing Department of the German Cancer Research Center. Its core idea is to automatically analyze the training data and configure the optimal U-Net architecture and training strategy for each specific segmentation task, thereby achieving excellent segmentation performance without expert intervention.) segmentation of rectal cancer T2 weighted imaging (T2 weighted The study used T2WI (T2WI) images to identify tumor areas and extracted radiomic features based on these images to predict regional lymph node metastasis in rectal cancer. However, this study only used T2WI, which reflects the morphological characteristics of the tumor, and did not combine images that can reflect the functional characteristics of the tumor, such as diffusion-weighted imaging (DWI). In addition, the rectal cancer segmentation in this study was completed based on nnUnetV1, while the currently updated nnUnetV2 (nnUNet V2 maintains the core functions of V1 and significantly improves flexibility and efficiency through improvements such as hierarchical labeling, cross-platform support, data format expansion, and storage optimization) can use fewer computing resources to train the network more efficiently and can be applied across multiple platforms, improving the efficiency and practicality of the model. Summary of the Invention

[0004] In view of the shortcomings and deficiencies of the existing technology, the present invention proposes a method for predicting the status of regional lymph nodes in rectal cancer based on nnUnetV2 and radiomics, which specifically includes the following steps:

[0005] Step 1: Data collection;

[0006] Preoperative magnetic resonance imaging (MRI) of patients with rectal cancer should be obtained, including oblique axial small-field-of-view high-resolution T2WI and axial single-shot echo-planar imaging (DWI). Pathologists should embed all lymph nodes detected in the postoperative specimen. Larger lymph nodes should be dissected and embedded. At least 12 lymph nodes should be detected in radical resection specimens without neoadjuvant therapy. Embedded lymph nodes should then be sectioned and stained with hematoxylin and eosin (HE). The sections should be microscopically examined for the presence of cancer cells to determine whether metastatic or non-metastatic lymph nodes are present.

[0007] Magnetic resonance imaging was acquired using an MR scanner; oblique axial small-field-of-view high-resolution fast spin-echo T2WI sequences were perpendicular to the long axis of the rectum where the tumor was located, and axial single-shot echo planar imaging (DWI) sequences were parallel to the horizontal line.

[0008] In the case dataset, the inclusion criteria for preoperative images of patients with rectal cancer were: 1) presence of pathologically confirmed rectal adenocarcinoma; 2) pelvic oblique axial small field of view high-resolution T2WI and conventional DWI examination; 3) postoperative pathological confirmation of regional lymph node status of rectal cancer; exclusion criteria were: 1) other types of rectal cancer, including mucinous adenocarcinoma, neuroendocrine carcinoma, and malignant melanoma; 2) poor image quality; 3) tumor volume less than A cm 3 ;4) history of pelvic surgery;

[0009] Step 2: VOI delineation of the rectal cancer tumor area;

[0010] VOI is a region of interest. When it comes to radiomics research, this region is semi-automatically or manually delineated using professional software. VOI represents a three-dimensional area.

[0011] In addition to the tumor area, the adjacent normal rectal wall and intestinal cavity area are outlined on the T2WI image, and the tumor area, adjacent normal rectal wall, and intestinal cavity are distinguished by marking; the magnetic susceptibility artifact area is marked on the DWI image; and the VOI of the entire cancer lesion is obtained;

[0012] Step 3: MRI image preprocessing;

[0013] Image preprocessing techniques were used to obtain standardized MRI images; image resampling and cropping were performed to ensure area size matching; grayscale value distribution was fine-tuned to linearly normalize grayscale values within the range of (0, 1); and image enhancement methods were used to amplify rectal cancer image data.

[0014] Step 4: Automatic segmentation model training;

[0015] The standardized T2WI and DWI images are input into the nnU-netV2 deep learning network, and the VOI is manually outlined as the real value for model training and verification; the architecture of the nnU-Net network includes an input layer, an encoder, a skip connection layer, a decoder, and an output layer; the input layer is used to accept image input, and the number of input channels is determined by the data type; the encoder contains a downsampling path composed of multiple convolutions Conv+normalization Norm+activation ReLU+pooling, which gradually extracts features and reduces resolution; the skip connection part establishes a connection between the corresponding encoding layer and the decoding layer to maintain high-resolution information; the decoder part adopts an upsampling path composed of deconvolution Transposed Conv+convolution Conv to restore features and improve resolution; the output layer Output Layer outputs the segmentation result after the final convolution layer;

[0016] The network was trained using a combination of the Dice coefficient and cross-entropy loss function, and the network was iteratively optimized using the stochastic gradient descent (SGD) optimizer. A piecewise learning rate decay strategy was used, and deep supervision was enabled to enhance training effectiveness through network layers. The trained model was used for automatic segmentation of tumor regions of interest, and the performance of the nnU-netV2 segmentation model was evaluated using the Dice coefficient, which ranges from 0 to 1, with 1 indicating complete overlap and 0 indicating no overlap.

[0017] The cross entropy loss function is as follows:

[0018]

[0019] Where N is the total number of pixels, y i is the actual label, p i is the predicted probability;

[0020] The loss function of the Dice coefficient is as follows:

[0021]

[0022] Among them, X is the prediction result of the model and Y is the actual label;

[0023] The formula of the stochastic gradient descent SGD algorithm is as follows:

[0024]

[0025] Among them, θ t represents the model parameters at the tth iteration, η represents the learning rate, Indicates that a single sample x is used at the tth iterationt The calculated gradient;

[0026] The Dice coefficient calculation formula is:

[0027]

[0028] Step 5: Radiomics feature extraction;

[0029] Six datasets of different modalities were established, namely T2WI+MbV-manual VOI segmentation, T2WI+DbV-automatic VOI segmentation, DWI+MbV, DWI+DbV, T2WI+DWI+MbV, and T2WI+DWI+DbV. Radiomic features were extracted based on MbV or DbV for T2WI and DWI original images and images after filter transformation. Radiomic features include: morphological features (shap); first-order features (firstorder); gray-level co-occurrence matrix features (glcm); gray-level run-length matrix features (glrlm); gray-level size region matrix features (glszm); gray-level dependency matrix features (gldm); and adjacent gray-level difference matrix features (ngtdm). Multiple features were extracted from each image sequence.

[0030] Step 6: Feature dimensionality reduction and selection;

[0031] Person correlation analysis was used to exclude highly repetitive features with correlation coefficients greater than B. The LASSO regression objective function and the mRMR objective function were used to identify the optimal features for model construction in a single-modality dataset. The LASSO regression objective function introduced an L1 regularization term to make the coefficients of some unimportant features zero, thereby achieving optimal feature selection. The mRMR objective function aims to maximize the correlation between features and the target variable while minimizing the redundancy between features to effectively screen out the optimal features. When using the recursive feature elimination (RFE) objective function to screen multimodal datasets (T2WI+DWI+MbV and T2WI+DWI+DbV), the most influential features were found by recursively reducing the size of the feature set.

[0032] The LASSO regression objective function is as follows:

[0033]

[0034] Among them, n is the number of samples corresponding to each of the six data sets in step 5, and p is the number of weights; a i is the feature vector of the i-th sample input, b i is the actual output of the i-th sample, and w represents the weight vector; is the mean square error, which is used to measure the predicted value w T a i and the actual value bi The error between them, T represents the transpose of the weight vector w; is the norm regularization term, λ is the regularization parameter; w j represents the jth weight in the weight vector w;

[0035] The mRMR objective function is as follows:

[0036]

[0037] Among them, S represents the feature subset after LASSO regression screening, p i is the feature after LASSO regression screening, I(p i ;q) represents feature p i The mutual information between the feature and the target variable q is used to measure the dependency between the two. The larger the value, the stronger the correlation between the feature and the target variable. i ;p j ) represents feature p i With p j Mutual information between features is used to measure the degree of redundancy, mutual influence or information overlap between selected features;

[0038] The RFE objective function is as follows:

[0039]

[0040] in, represents the RFE objective function, Represents the objective function for p i The partial derivative of Represents the objective function for p j The partial derivative of

[0041] Step 7: Use logistic regression (LR) to build a prediction model to predict lymph node status;

[0042] Logistic regression LR was used to construct a prediction model to predict lymph node status;

[0043] The area under the receiver operating characteristic (ROC) curve (AUC) was calculated to evaluate the performance of the model in predicting regional lymph node status. The AUC value ranged from 0 to 1. The larger the value, the better the model performance. This was used to perform the final lymph node prediction.

[0044] The AUC value is obtained by dividing the false positive rate by the true positive rate:

[0045]

[0046] Among them, G represents the number of positive samples, Q represents the number of negative samples, positive samples are samples with metastatic lymph nodes, and negative samples are samples without metastatic lymph nodes. Postoperative pathological examination is used to determine whether they are positive samples or negative samples. f(x i ) represents the radiomics model for the positive sample x i The prediction of f(x j ) represents the radiomics model for negative samples x j The predicted output, Ⅱ(f(x i )>f(x j ) is the indicator function, when f(x i )>f(x j ) takes the value of 1, otherwise it takes the value of 0; the Delong test is used to perform the significance test of AUC, which is achieved by calculating the z value:

[0047]

[0048] Among them, AUC1 and AUC2 represent the AUC values of the two models, respectively. “Two models” refers to any two lymph node prediction models. var(AUC1) and var(AUC2) represent the variances of the two AUC values, respectively. cov(AUC1, AUC2) represents the covariance of the two AUC values.

[0049] In a specific embodiment of the present invention, in step 1, the MR scanner uses an 8-channel abdominal coil to acquire image signals; the detailed scanning parameters of the T2WI sequence are as follows: fast spin echo sequence TSE, TR / TE=5000 / 130 (m / s), slice thickness=3 mm, slice spacing=1 mm, FOV=20 cm, matrix=352×256, number of excitations=2; the detailed scanning parameters of the DWI sequence are as follows: planar echo sequence EPI, TR / TE=4700 / Min Full (m / s), slice thickness=3 mm, slice spacing=1 mm, FOV=34 cm, matrix=128×140, number of excitations=8, b=0, 1000 (mm 2 / s).

[0050] In step 1 of one embodiment of the present invention, the exclusion criteria for preoperative images of rectal cancer patients collected are: 2) poor image quality with severe artifacts or deformation; 3) tumor volume less than 1 cm 3 .

[0051] In step 2 of another embodiment of the present invention, the rectal cancer VOI is manually delineated layer by layer by a radiologist using ITK-SNAP software on T2WI and DWI images to define the area where the tumor is located.

[0052] In yet another embodiment of the present invention, step 3 is specifically as follows:

[0053] Step 1: Use Python's pydicom and nibabel libraries to convert T2WI and DWI image formats from dicom to nifti;

[0054] Step 2: Use image preprocessing technology to obtain standardized MRI images;

[0055] Step 3: Resample and crop the images to ensure that the area sizes match, so that the T2WI and DWI image matrices are 128 × 128 × 32 and the spatial resolution is 0.4 × 0.4 × 3.3 mm.

[0056] Step 4: Fine-tune the grayscale value distribution so that the grayscale value is linearly normalized within the range of (0, 1);

[0057] Step 5: Use rotation and slight flipping to enhance the image data and amplify the rectal cancer image data.

[0058] In step 4 of another specific embodiment of the present invention, the learning rate is set to 0.01, the momentum is set to 0.99, and the weight decay is set to 3e-05; the batch size of the piecewise decay is set to 2.

[0059] In step 5 of another specific embodiment of the present invention, the Pyradiomics plug-in in the Python software is used to extract imaging omics features based on MbV or DbV for the T2WI and DWI original images and the images after filter transformation; 1037 features will be extracted from each single data set, including: First-Order: 18; Morphological Shape: 14; Gray-level co-occurrence matrix GLCM: 24; Gray-level dependency matrix GLDM: 14; Gray-level run matrix GLRLM: 16; Gray-level size region matrix GLSZM: 16; Neighboring grayscale difference matrix NGTDM: 5; Laplacian operator feature Lbp-3D: 186; Wavelet filter feature Wavelet: 743; This is a total of 1036, plus the pyradiomics version number, a total of 1037 features.

[0060] In another specific embodiment of the present invention, the filter is a logarithmic LoG and wavelet filter.

[0061] In addition, in step 6 of a specific embodiment of the present invention, person correlation analysis is used to exclude highly repetitive features with correlation coefficients greater than 0.9.

[0062] The present invention combines the automatic segmentation based on deep learning nnU-NetV2 with MRI imaging genomics features to solve the problem of time-consuming and inefficient manual labeling by doctors in traditional imaging genomics methods. It can also extract imaging genomics features that better reflect the actual situation of training samples from medical images of different modalities, providing auxiliary diagnosis for non-invasive prediction of regional lymph node status in patients with rectal cancer, which is conducive to the accurate prediction of lymph nodes and provides more accurate and personalized diagnosis and treatment plans for colorectal cancer. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 A flow chart showing the method of the present invention;

[0064] Figure 2 Shown is a schematic diagram of the structure of nnU-netV2;

[0065] Figure 3 Schematic diagram showing the manually outlined area on T2WI and DWI images and the automatic segmentation of the tumor area based on nnU-netV2;

[0066] Figure 4 The ROC diagram after the model execution in the present invention is shown. DETAILED DESCRIPTION

[0067] The present invention will be described in detail below with reference to the accompanying drawings.

[0068] This paper provides a method for predicting regional lymph node status in rectal cancer using deep learning and radiomics based on MRI images. This method uses nnUnetV2 to segment tumor regions in rectal cancer T2WI and DWI images, and establishes a method for predicting regional lymph nodes in rectal cancer based on radiomics features extracted from the automatically segmented tumor regions. This method will improve the daily work efficiency of imaging physicians and provide a basis for clinicians to specify personalized treatment strategies. The method specifically includes the following steps:

[0069] Step 1: Data collection;

[0070] Preoperative magnetic resonance imaging (MRI) images, including T2WI (oblique axial small field of view high resolution) and DWI (axial single echo planar imaging), are collected from patients with rectal cancer. Pathologists embed all lymph nodes detected in the postoperative specimen. Larger lymph nodes should be dissected and embedded. At least 12 lymph nodes should be identified in radical resection specimens without neoadjuvant therapy. Embedded lymph nodes are then sectioned and stained with hematoxylin-eosin (HE). The sections are then microscopically examined for the presence of cancer cells to determine whether they are metastatic or non-metastatic lymph nodes.

[0071] Magnetic resonance imaging (MRI) was acquired using a 3.0T MR scanner (Discovery MR750, GE Medical Systems, USA; Siemens MAGNETOM TrioTim, Siemens Healthineers, Germany). An 8-channel abdominal coil was used for image acquisition. An oblique axial, small-field-of-view, high-resolution, fast spin-echo (T2WI) sequence was perpendicular to the long axis of the rectum where the tumor was located, while an axial, single-shot echo-planar imaging (DWI) sequence was parallel to the horizontal. Detailed T2WI scanning parameters were as follows: fast spin-echo (TSE), TR / TE (m / s) = 5000 / 130, slice thickness = 3 mm, interslice spacing = 1 mm, field of view (FOV) = 20 cm, matrix = 352 × 256, and number of excitations = 2. Detailed DWI scanning parameters: planar echo (EPI) sequence, TR / TE (m / s) = 4700 / Min Full, slice thickness = 3 mm, slice spacing = 1 mm, FOV = 34 cm, matrix = 128 × 140, number of excitations = 8, b = 0, 1000 mm 2 / s.

[0072] In the case dataset, the inclusion criteria for preoperative images of rectal cancer patients were as follows: 1) the presence of rectal adenocarcinoma confirmed by pathology (biopsy or surgery); 2) pelvic T2WI (oblique axial small field of view high resolution) and conventional DWI (b = 0, 1000 mm 2 3) Postoperative pathological confirmation of regional lymph node status of rectal cancer. Exclusion criteria were: 1) other types of rectal cancer, including mucinous adenocarcinoma, neuroendocrine carcinoma, and malignant melanoma; 2) poor image quality (severe artifacts or deformation); 3) tumor volume less than 1 cm 3 ;4) History of pelvic surgery.

[0073] Step 2: delineation of the rectal cancer tumor area (volume of interest, VOI);

[0074] VOI stands for region of interest. When it comes to radiomics research, a professional physician needs to identify and delineate the target area based on their personal experience. This area is then semi-automatically or manually delineated using specialized software (such as ITK-SNAP). VOI typically represents a three-dimensional region.

[0075] In medicine, two-dimensional contouring only delineates the largest layer of a tumor. Three-dimensional region contouring is now more commonly used. This technique typically involves delineating the tumor layer by layer on an axial image, either from head to toe or from toe to head. Contouring software (such as ITK-SNAP) then automatically stitches together a three-dimensional volume of interest (VOI).

[0076] In one specific implementation of the present invention, a radiologist with extensive abdominal imaging experience manually delineated the tumor region layer by layer on T2WI and DWI images using ITK-SNAP software. This was then reviewed by another radiologist with extensive experience. The delineation results from different experienced radiologists were generally consistent, and slight differences did not affect subsequent analysis and resolution.

[0077] In addition to the tumor area, the present invention also requires manual delineation of the adjacent normal rectal wall and intestinal cavity area on the T2WI image. The tumor area is marked in red, the adjacent normal rectal wall is marked in green, and the intestinal cavity is marked in blue (see Figure 2 ). As above, the results of the delineation by different technicians with rich experience in this field are roughly the same, and slight differences do not affect the subsequent analysis and solution of the problem. Even if the present invention partially involves human experience, as long as the delineation is performed by technicians with rich experience in this field, the results that the present invention wants to obtain can be obtained, and individual differences do not affect the final result of the problem solved by the present invention. The magnetically sensitive artifact area must also be marked on the DWI. The magnetically sensitive artifact area is marked in yellow, for example (there is no magnetically sensitive artifact area in the example figure). Finally, the VOI of the entire cancer lesion is obtained. Although the above-mentioned marking method is well known to those skilled in the art, the above-mentioned marking method of the present invention is different from other studies. Other studies generally only mark the tumor area, and do not mark the area outside the tumor.

[0078] Step 3: MRI image preprocessing;

[0079] Step 1: Use Python's pydicom and nibabel libraries to convert T2WI and DWI image formats from dicom to nifti (this method is well known to those skilled in the art);

[0080] Step 2: Use image preprocessing technology to obtain standardized MRI images (this method is well known to those skilled in the art);

[0081] Step 3: Perform image resampling and cropping (this method is well known to those skilled in the art) to ensure that the area sizes match. In a specific embodiment of the present invention, the T2WI and DWI image matrices are sized to 128 × 128 × 32, with a spatial resolution of 0.4 × 0.4 × 3.3 mm.

[0082] Step 4: Fine-tune the grayscale value distribution so that the grayscale value is linearly normalized within the range of (0, 1);

[0083] Step 5: Amplify the rectal cancer image data using an image enhancement method, including rotation and slight flipping, which is well known to those skilled in the art.

[0084] Step 4: Automatic segmentation model training;

[0085] The standardized T2WI and DWI images in step 3 are input into the nnU-netV2 deep learning network, and the VOI is manually outlined as the real value for model training and verification. Figure 2 ), including input layer, encoder, skip connection layer, decoder and output layer. The input layer (Input Layer) is mainly used to accept image input, and the number of input channels is determined by the data type (3D image input in this study). The encoder (Encoder) contains a downsampling path composed of multiple convolution (Conv) + normalization (Norm) + activation (ReLU) + pooling (Pooling), which gradually extracts features and reduces resolution. The skip connection (Skip Connections) part establishes a connection between the corresponding encoding layer and decoding layer to maintain high-resolution information. The decoder (Decoder) part uses an upsampling path composed of deconvolution (Transposed Conv) + convolution (Conv) to restore features and improve resolution. The output layer (Output Layer) outputs the segmentation result after the final convolution layer. In this implementation, the Dice coefficient (Sorensen T (1948) A method of establishing groups of equal amplitude in plant sociology based on similarity of species content and its application to analyses of the vegetation on Danish commons. https: / / api.semanticscholar.org / CorpusID:135206594) and the cross-entropy loss function were combined for network training. The network was iteratively optimized using the Stochastic Gradient Descent (SGD) optimizer, with a learning rate of 0.01, momentum of 0.99, and weight decay of 3e-05. The learning rate decay strategy was piecewise, the batch size was set to 2 (two samples were used in each iteration to calculate the gradient and update the parameters), and deep supervision was enabled to enhance training across network layers. The trained model was used for automatic segmentation of tumor regions of interest, and the performance of the nnU-netV2 segmentation model was evaluated using the Dice coefficient, which ranges from 0 to 1, with 1 indicating complete overlap and 0 indicating no overlap.

[0086] The cross entropy loss function is as follows:

[0087]

[0088] Where N is the total number of pixels in the image, y i is the actual label, p i is the predicted probability.

[0089] The loss function of the Dice coefficient is as follows:

[0090]

[0091] Among them, X is the prediction result of the model and Y is the actual label.

[0092] The SGD algorithm formula is as follows:

[0093]

[0094] Among them, θ t represents the model parameters at the tth iteration, η represents the learning rate, Indicates that a single sample x is used at the tth iteration t The calculated gradient is known to those skilled in the art.

[0095] The Dice coefficient calculation formula is:

[0096]

[0097] An example of tumor area automatically segmented based on the nnU-netV2 deep learning network is shown in Figure 3 ,The red marked areas on the T2WI and DWI images of patients 1 and 2 are all tumor areas.

[0098] Step 5: Radiomics feature extraction;

[0099] Six data sets of different modalities were established, namely T2WI+MbV (manual VOI segmentation), T2WI+DbV (automatic VOI segmentation), DWI+MbV, DWI+DbV, T2WI+DWI+MbV, and T2WI+DWI+DbV. The Pyradiomics plug-in in Python software is used to transform the T2WI and DWI original images and the images after filter (logarithm (Log) and wavelet (Wavelet)) transformation (this transformation is implemented by the Pyradiomics plug-in in Python software, and the method of extracting imaging omics features is well known to those skilled in the art) based on MbV or DbV to extract imaging omics features (for example, using the Pyradiomics plug-in in Python software, which is well known to those skilled in the art); imaging omics features include: morphological features (shap); first-order features (firstorder); grayscale co-occurrence matrix features (glcm); grayscale run-length matrix features (glrlm); grayscale size region matrix features (glszm); grayscale dependence matrix features (gldm); adjacent grayscale difference matrix features (ngtdm); multiple features are extracted from each image sequence.

[0100] In a specific embodiment of the present invention, because the names of the above-mentioned imaging features are fixed in the Pyradiomics plug-in, 1037 features will be extracted when extracting these categories of imaging omics features, including: First-Order: 18; Morphology Shape: 14; Gray Level Co-occurrence Matrix GLCM: 24; Gray Level Dependence Matrix GLDM: 14; Gray Level Run Matrix GLRLM: 16; Gray Level Size Region Matrix GLSZM: 16; Neighboring Gray Level Difference Matrix NGTDM: 5; Laplacian Operator Features Lbp-3D: 186; Wavelet Filter Features Wavelet: 743; This is a total of 1036, plus the pyradiomics version number column, a total of 1037 features.

[0101] Step 6: Feature dimensionality reduction and selection;

[0102] Person correlation analysis (Tibshirani, Robert. "The lasso method for variable selection in the Cox model." Statistics in medicine 16.4 (1997): 385-395.) was used to exclude highly repetitive features with a correlation coefficient greater than 0.9. The LASSO regression objective function and the mRMR objective function were used to identify the optimal features for building a model in a unimodal data set. Among them, the LASSO regression objective function introduces an L1 regularization term so that the coefficients of features with low correlation with the target variable or redundant features (these features are well known to those skilled in the art) are zero, thereby achieving optimal feature selection; the goal of the mRMR objective function is to minimize the redundancy between features while maximizing the correlation between the features and the target variable, thereby effectively screening out the optimal features. Especially when there are many features, it can help the model improve performance and reduce complexity. The above two techniques are well known to those skilled in the art. The recursive feature elimination (RFE) objective function (originally proposed by ISABELLE GUYON et al. in "Gene selection for cancer classification using support vector machines" in 2003) was used to screen for more important features in multimodal datasets (T2WI+DWI+MbV and T2WI+DWI+DbV). When using the RFE objective function method for feature selection, the most influential features are found by recursively reducing the size of the feature set. This process relies on the model itself to evaluate the importance of the features, and gradually removes the least important features based on the importance assigned by the model, and constructs the prediction label using the selected important features. This method is known to those skilled in the art.

[0103] The LASSO regression objective function is as follows:

[0104]

[0105] Where n is the number of samples corresponding to each of the six data sets in step 5, and p is the number of weights. i is an index, representing the sequence number of the sample, usually ranging from 1 to n (where n is the total number of training samples). i is the feature vector of the i-th sample input, b i is the actual output of the i-th sample, and w is the regression coefficient vector. is the mean square error, which is used to measure the predicted value w T a i and the actual value b i The error between . is the norm regularization term, λ is the regularization parameter, T represents the transpose of the weight vector w. w represents the weight vector, w j Represents the weight vector w j The jth weight in .

[0106] The mRMR objective function is as follows:

[0107]

[0108] Among them, S represents the feature subset after LASSO regression screening, I(x i ; y) represents feature p i The mutual information between the target variable q, p i It is obtained from the features filtered by LASSO regression. Mutual information is used to measure the dependency between the two. The larger the value, the stronger the correlation between the feature and the target variable. i ;p j ) represents feature p i With p j The mutual information between them is used to measure the redundancy, mutual influence or information overlap between selected features.

[0109] The RFE objective function is as follows:

[0110]

[0111] in, represents the RFE objective function, Represents the objective function for p i The partial derivative of Represents the objective function for p j The partial derivative of .

[0112] The cross entropy loss function evaluates the probability p predicted by the model i and the actual label y i The degree of matching between them is often used in classification problems, especially binary classification problems.

[0113] RFE loss function: The goal of RFE is to optimize performance by selecting features that influence model performance. The specific form of its loss function depends on the base model used. By calculating the partial derivatives (i.e., gradients) of the loss function with respect to the features, RFE can determine the degree of influence of each feature on the model output. Features with large partial derivatives are more likely to influence the model results, while features with partial derivatives close to zero may be considered unimportant. In this way, unimportant features are gradually eliminated.

[0114] These two loss functions are different. Although they are both related to model evaluation, their purpose and specific form are different.

[0115] Step 7: Use logistic regression (LR) to build a prediction model to predict lymph node status;

[0116] The study used five machine learning algorithms—logistic regression (LR), support vector machine (SVM), random forest (RF), k-nearest neighbor (KNN), and classification and regression tree (CART)—to build a predictive model for lymph node status. Logistic regression (LR) was found to be the most effective, so LR was used to build a predictive model based on the six modalities.

[0117] The area under the receiver operating characteristic (ROC) curve (AUC) was calculated to evaluate the performance of the model in predicting regional lymph node status. The imaging omics model with the best performance was selected from the datasets of the six modalities by comparing the AUC values of each model. The larger the AUC value, the better the model performance was. The AUC value was used to perform the final lymph node prediction. The AUC value ranged from 0 to 1. The larger the value, the higher the prediction performance of the imaging omics model.

[0118] The AUC value is obtained by dividing the false positive rate by the true positive rate:

[0119]

[0120] Among them, P represents the number of positive samples, Q represents the number of negative samples (positive samples are samples with metastatic lymph nodes, and negative samples are samples without metastatic lymph nodes. Postoperative pathological examination is used to determine whether they are positive or negative samples). i ) represents the radiomics model for the positive sample x i The prediction of f(x j ) represents the radiomics model for negative samples x j The predicted output, Ⅱ(f(x i )>f(x j ) is the indicator function, when f(x i )>f(x j) takes the value of 1, otherwise it takes the value of 0. The Delong test (this method was first proposed by Elizabeth H. DeLong et al. in their 1988 article "Comparing the Areas Under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach". In subsequent practical applications, Sun Xu et al.'s "Fast Implementation of DeLong's Algorithm for Comparing the Areas Under Correlated Receiver Operating Characteristic Curves" further optimized the original algorithm and improved computational efficiency, and is well known to those skilled in the art) is used to perform the significance test of AUC, which is achieved by calculating the z value:

[0121]

[0122] Among them, AUC1 and AUC2 represent the AUC values of the two models respectively ("two models" refer to any two lymph node prediction models, which are the models constructed above. Six prediction models can be constructed based on image datasets of different modalities using logistic regression LR), var(AUC1) and var(AUC2) represent the variances of the two AUC values respectively, and cov(AUC1, AUC2) represents the covariance of the two AUC values.

[0123] Figure 4In the figure, MbV represents the model constructed based on radiomic features obtained from manually segmented VOIs, and DbV represents the model constructed based on radiomic features obtained from automatically segmented VOIs using nnU-NetV2. This figure shows that the performance of the radiomics model based on deep learning automatic segmentation in this study is comparable to that of the radiomics model based on manual segmentation, whether based on single-modality or multimodal data. This further validates the high accuracy and excellent robustness of nnU-netV2 in tumor segmentation tasks. Furthermore, the radiomics model based on multimodal features (T2WI+DWI+DbV (nnU-netV2)) outperformed models based solely on T2WI imaging features (AUC = 0.700) or DWI imaging features (AUC = 0.667) in predicting metastatic lymph nodes (validation set, AUC = 0.800). This demonstrates that multimodal data can improve the accuracy and robustness of model prediction by integrating and cross-validating information obtained from multiple imaging modalities. The present invention automatically segments the tumor area of rectal cancer T2WI and DWI images based on nnUnetV2, and establishes a rectal cancer regional lymph node prediction method based on the imaging genomics features extracted from the automatically segmented tumor area. This will improve the daily work efficiency of imaging doctors and provide a basis for clinicians to formulate personalized treatment strategies.

Claims

1. A method for predicting regional lymph node status in rectal cancer based on nnUnetV2 and radiomics, characterized in that: The specific steps include: Step 1: Data collection; Preoperative magnetic resonance imaging (MRI) of patients with rectal cancer should be obtained, including oblique axial small-field-of-view high-resolution T2WI and axial single-shot echo-planar imaging (DWI). Pathologists should embed all lymph nodes detected in the postoperative specimen. Larger lymph nodes should be dissected and embedded. At least 12 lymph nodes should be detected in radical resection specimens without neoadjuvant therapy. Embedded lymph nodes should then be sectioned and stained with hematoxylin and eosin (HE). The sections should be microscopically examined for the presence of cancer cells to determine whether metastatic or non-metastatic lymph nodes are present. Magnetic resonance imaging was acquired using an MR scanner; oblique axial small-field-of-view high-resolution fast spin-echo T2WI sequences were perpendicular to the long axis of the rectum where the tumor was located, and axial single-shot echo planar imaging (DWI) sequences were parallel to the horizontal line. In the case dataset, the inclusion criteria for preoperative images of patients with rectal cancer were: 1) presence of pathologically confirmed rectal adenocarcinoma; 2) pelvic oblique axial small field of view high-resolution T2WI and conventional DWI examination; 3) postoperative pathological confirmation of regional lymph node status of rectal cancer; exclusion criteria were: 1) other types of rectal cancer, including mucinous adenocarcinoma, neuroendocrine carcinoma, and malignant melanoma; 2) poor image quality; 3) tumor volume less than A cm 3 ;4) history of pelvic surgery; Step 2: VOI delineation of the rectal cancer tumor area; VOI is a region of interest. When it comes to radiomics research, this region is semi-automatically or manually delineated using professional software. VOI represents a three-dimensional area. In addition to the tumor area, the adjacent normal rectal wall and intestinal cavity area were outlined on the T2WI image, and the tumor area, adjacent normal rectal wall, and intestinal cavity were marked; the magnetic susceptibility artifact area was marked on the DWI image; Obtain the VOI of the entire cancer lesion; Step 3: MRI image preprocessing; Image preprocessing techniques were used to obtain standardized MRI images; image resampling and cropping were performed to ensure area size matching; grayscale value distribution was fine-tuned to linearly normalize grayscale values within the range of (0, 1); and image enhancement methods were used to amplify rectal cancer image data. Step 4: Automatic segmentation model training; The standardized T2WI and DWI images are input into the nnU-netV2 deep learning network, and the VOI is manually outlined as the real value for model training and verification; the architecture of the nnU-Net network includes an input layer, an encoder, a skip connection layer, a decoder, and an output layer; the input layer is used to accept image input, and the number of input channels is determined by the data type; the encoder contains a downsampling path composed of multiple convolutions Conv+normalization Norm+activation ReLU+pooling, which gradually extracts features and reduces resolution; the skip connection part establishes a connection between the corresponding encoding layer and the decoding layer to maintain high-resolution information; the decoder part adopts an upsampling path composed of deconvolution Transposed Conv+convolution Conv to restore features and improve resolution; the output layer Output Layer outputs the segmentation result after the final convolution layer; The network was trained using a combination of the Dice coefficient and cross-entropy loss function, and the network was iteratively optimized using the stochastic gradient descent (SGD) optimizer. A piecewise learning rate decay strategy was used, and deep supervision was enabled to enhance training effectiveness through network layers. The trained model was used for automatic segmentation of tumor regions of interest, and the performance of the nnU-netV2 segmentation model was evaluated using the Dice coefficient, which ranges from 0 to 1, with 1 indicating complete overlap and 0 indicating no overlap. The cross entropy loss function is as follows: Where N is the total number of pixels, y i is the actual label, p i is the predicted probability; The loss function of the Dice coefficient is as follows: Among them, X is the prediction result of the model and Y is the actual label; The formula of the stochastic gradient descent SGD algorithm is as follows: Among them, θ t represents the model parameters at the tth iteration, η represents the learning rate, Indicates that a single sample x is used at the tth iteration t The calculated gradient; The Dice coefficient calculation formula is: Step 5: Radiomics feature extraction; Six datasets of different modalities were established, namely T2WI+MbV-manual VOI segmentation, T2WI+DbV-automatic VOI segmentation, DWI+MbV, DWI+DbV, T2WI+DWI+MbV, and T2WI+DWI+DbV. Radiomic features were extracted based on MbV or DbV for T2WI and DWI original images and images after filter transformation. Radiomic features include: morphological features (shap); first-order features (firstorder); gray-level co-occurrence matrix features (glcm); gray-level run-length matrix features (glrlm); gray-level size region matrix features (glszm); gray-level dependency matrix features (gldm); and adjacent gray-level difference matrix features (ngtdm). Multiple features were extracted from each image sequence. Step 6: Feature dimensionality reduction and selection; Person correlation analysis was used to exclude highly repetitive features with correlation coefficients greater than B. The LASSO regression objective function and the mRMR objective function were used to identify the optimal features for model construction in a single-modality dataset. The LASSO regression objective function introduced an L1 regularization term to make the coefficients of some unimportant features zero, thereby achieving optimal feature selection. The mRMR objective function aims to maximize the correlation between features and the target variable while minimizing the redundancy between features to effectively screen out the optimal features. When using the recursive feature elimination (RFE) objective function to screen multimodal datasets (T2WI+DWI+MbV and T2WI+DWI+DbV), the most influential features were found by recursively reducing the size of the feature set. The LASSO regression objective function is as follows: Among them, n is the number of samples corresponding to each of the six data sets in step 5, and p is the number of weights; a i is the feature vector of the i-th sample input, b i is the actual output of the i-th sample, and w represents the weight vector; is the mean square error, which is used to measure the predicted value w T a i and the actual value b i The error between them, T represents the transpose of the weight vector w; is the norm regularization term, λ is the regularization parameter; w j represents the jth weight in the weight vector w; The mRMR objective function is as follows: Among them, S represents the feature subset after LASSO regression screening, p i is the feature after LASSO regression screening, I(p i ;q) represents feature p i The mutual information between the feature and the target variable q is used to measure the dependency between the two. The larger the value, the stronger the correlation between the feature and the target variable. i ;p j ) represents feature p i With p j Mutual information between features is used to measure the degree of redundancy, mutual influence or information overlap between selected features; The RFE objective function is as follows: in, represents the RFE objective function, Represents the objective function for p i The partial derivative of Represents the objective function for p j The partial derivative of Step 7: Use logistic regression (LR) to build a prediction model to predict lymph node status; Logistic regression LR was used to construct a prediction model to predict lymph node status; The area under the receiver operating characteristic (ROC) curve (AUC) was calculated to evaluate the performance of the model in predicting regional lymph node status. The AUC value ranged from 0 to 1. The larger the value, the better the model performance. This was used to perform the final lymph node prediction. The AUC value is obtained by dividing the false positive rate by the true positive rate: Among them, G represents the number of positive samples, Q represents the number of negative samples, positive samples are samples with metastatic lymph nodes, and negative samples are samples without metastatic lymph nodes. Postoperative pathological examination is used to determine whether they are positive samples or negative samples. f(x i ) represents the radiomics model for the positive sample x i The prediction of f(x j ) represents the radiomics model for negative samples x j The predicted output, Ⅱ(f(x i )>f(x j ) is the indicator function, when f(x i )>f(x j ) takes the value of 1, otherwise it takes the value of 0; the Delong test is used to perform the significance test of AUC, which is achieved by calculating the z value: Where AUC1 and AUC2 represent the AUC values of the two models, respectively. "Two models" refers to any two lymph node prediction models. var(AUC1) and var(AUC2) represent the variances of the two AUC values, respectively. cov(AUC1, AUC2) represents the covariance of the two AUC values.

2. The method for predicting regional lymph node status of rectal cancer based on nnUnetV2 and radiomics according to claim 1, characterized in that: In step 1, the MR scanner used an 8-channel abdominal coil to acquire image signals. Detailed T2WI scanning parameters were as follows: fast spin echo (TSE), TR / TE = 5000 / 130 (m / s), slice thickness = 3 mm, slice spacing = 1 mm, field of view (FOV) = 20 cm, matrix = 352 × 256, and number of excitations = 2. Detailed scanning parameters of the DWI sequence: planar echo sequence EPI, TR / TE = 4700 / Min Full (m / s), slice thickness = 3 mm, slice spacing = 1 mm, FOV = 34 cm, matrix = 128 × 140, number of excitations = 8, b = 0, 1000 (mm 2 / s).

3. The method for predicting regional lymph node status of rectal cancer based on nnUnetV2 and radiomics according to claim 1, wherein: In step 1, the exclusion criteria for preoperative images of rectal cancer patients collected were: 2) poor image quality with severe artifacts or deformation; 3) tumor volume less than 1 cm 3 .

4. The method for predicting regional lymph node status of rectal cancer based on nnUnetV2 and radiomics according to claim 1, wherein: In step 2, the rectal cancer VOI was manually delineated layer by layer on T2WI and DWI images by radiologists using ITK-SNAP software.

5. The method for predicting regional lymph node status of rectal cancer based on nnUnetV2 and radiomics according to claim 1, wherein: Step 3 is as follows: Step 1: Use Python's pydicom and nibabel libraries to convert T2WI and DWI image formats from dicom to nifti; Step 2: Use image preprocessing technology to obtain standardized MRI images; Step 3: Resample and crop the images to ensure that the area sizes match, so that the T2WI and DWI image matrices are 128 × 128 × 32 and the spatial resolution is 0.4 × 0.4 × 3.3 mm. Step 4: Fine-tune the grayscale value distribution so that the grayscale value is linearly normalized within the range of (0, 1); Step 5: Use rotation and slight flipping to enhance the image data and amplify the rectal cancer image data.

6. The method for predicting regional lymph node status of rectal cancer based on nnUnetV2 and radiomics according to claim 1, wherein: In step 4, the learning rate is set to 0.01, the momentum is set to 0.99, and the weight decay is set to 3e-05; the batch size of the piecewise decay is set to 2.

7. The method for predicting regional lymph node status of rectal cancer based on nnUnetV2 and radiomics according to claim 1, wherein: In step 5, the Pyradiomics plug-in in Python software was used to extract radiomic features based on MbV or DbV from the original T2WI and DWI images and the images after filter transformation; 1037 features will be extracted from each single data set, including: First-Order: 18; Morphology: 14; Gray Level Co-occurrence Matrix GLCM: 24; Grayscale dependency matrix GLDM: 14; grayscale run-length matrix GLRLM: 16; grayscale size region matrix GLSZM: 16; adjacent grayscale difference matrix NGTDM: 5; Laplacian operator feature Lbp-3D: 186; wavelet filter feature Wavelet: 743; a total of 1036, plus the pyradiomics version number, a total of 1037 features.

8. The method for predicting regional lymph node status of rectal cancer based on nnUnetV2 and radiomics according to claim 7, wherein: The filters are logarithmic LoG and wavelet filters.

9. The method for predicting regional lymph node status of rectal cancer based on nnUnetV2 and radiomics according to claim 1, wherein: In step 6, person correlation analysis was used to exclude highly repetitive features with correlation coefficients greater than 0.9.

Images