Method and device for constructing colorectal cancer multi-mode recurrence risk model

Abstract

The invention relates to the technical field of digital medical treatment, in particular to a method and a device for constructing a colorectal cancer multi-mode recurrence risk model. The construction method comprises the following steps: acquiring a panoramic digital scanning image of a pathological tissue section of a colorectal cancer patient; constructing a data set and selecting a region of interest by using the panoramic digital scanning image; constructing a pathological omics feature scoring model, a collagen feature scoring model and an immune feature scoring model through the region of interest; and according to the pathological omics feature scoring model, the collagen feature scoring model and the immune feature scoring model, constructing a multi-modal recurrence risk model. On the basis of an image analysis technology, the pathological omics feature scoring model, the collagen feature scoring model and the immune feature scoring model are constructed and integrated into the multi-mode recurrence risk model, and accurate prediction of the recurrence risk of the colorectal cancer patient is achieved.

Description

Technical Field

[0001] The present invention relates to the field of digital medical technology, and in particular to a method and device for constructing a multimodal recurrence risk model for colorectal cancer. Background Art

[0002] In recent years, the incidence of colorectal cancer has increased year by year, and it has become the second most common and fourth most common malignant tumor in my country. The TNM staging system is the cornerstone of prognosis assessment and treatment decision-making for patients with colorectal cancer. However, even the latest 8th edition of the TNM staging system still has difficulty in accurately assessing the risk of recurrence in patients with colorectal cancer. Therefore, there is an urgent need to build a more accurate method to predict the risk of recurrence in patients with colorectal cancer, thereby promoting the development of personalized diagnosis and treatment.

[0003] In the prior art, a colorectal cancer collagen evaluation and prognosis prediction device and storage medium are disclosed. The computer-readable storage medium can image three areas: tumor tissue, invasion edge, and normal tissue, and extract collagen structural features. However, this method only considers a single collagen structure, ignores the effects of the morphological and structural characteristics of the tumor and its cells, and the number and location of immune cells on the risk of recurrence, and does not integrate multiple models to improve model accuracy. Therefore, if the pathological genomics feature scoring model, collagen feature scoring model, and immune feature scoring model can be integrated to construct an accurate and robust multimodal recurrence risk model, it will have important reference value in assisting individualized diagnosis and treatment decisions, improving the quality of life, and improving prognosis.

[0004] At present, there are not many studies on the multimodal recurrence risk model for colorectal cancer, and there is no specific multimodal recurrence risk model that integrates the pathological genomics feature scoring model, collagen feature scoring model, and immune feature scoring model. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention provides a method and device for constructing a multimodal recurrence risk model for colorectal cancer.

[0006] On the one hand, the construction method includes the following steps: obtaining a panoramic digital scanning image of a pathological tissue section of a colorectal cancer patient; using the panoramic digital scanning image, constructing a data set and selecting a region of interest; constructing a pathological genomics feature scoring model, a collagen feature scoring model, and an immune feature scoring model through the region of interest; constructing a multimodal recurrence risk model based on the pathological genomics feature scoring model, the collagen feature scoring model, and the immune feature scoring model. The present invention is based on the pathological tissue sections of colorectal cancer patients and provides an innovative means for predicting the risk of disease recurrence through a series of fine steps. The specific effects are as follows: First, through panoramic digital scanning images, the comprehensiveness and accuracy of pathological information are ensured, laying a solid foundation for subsequent analysis; second, by constructing data sets and outlining regions of interest, the efficiency and pertinence of data processing are effectively improved; third, by constructing pathological genomics feature scoring models, collagen feature scoring models and immune feature scoring models, the pathological characteristics are reflected in multiple dimensions, providing a scientific basis for accurately predicting the probability of colorectal cancer recurrence; fourth, by integrating pathological genomics, collagen features and immunological indicators to construct a multimodal recurrence risk model, it not only comprehensively considers multiple pathophysiological factors, but also significantly improves the accuracy and reliability of recurrence risk prediction, showing broad application prospects in the field of colorectal cancer recurrence risk assessment.

[0007] Optionally, the method of obtaining a panoramic digital scan image of a pathological tissue section of a colorectal cancer patient includes the following steps: obtaining a first type of pathological tissue section of a colorectal cancer patient; performing immunohistochemical staining on the first type of pathological tissue section to obtain a second type of pathological tissue section. Scanning the first type of pathological tissue section and the second type of pathological tissue section with a scanner to obtain a panoramic digital scan image. The present invention uses advanced scanner technology to perform a panoramic scan of two types of pathological tissue sections, thereby generating a high-quality panoramic digital scan image. The specific effects are as follows: First, by staining the pathological tissue sections, the specific antigens in the tissue are enhanced, making the cell structure and components clearer; second, by using high-resolution scanning equipment, every detail is not missed, which provides a solid foundation for subsequent data processing and model building, and ensures the consistency and reliability of the entire analysis process.

[0008] Optionally, the use of the panoramic digital scanning image to construct a data set and select a region of interest includes the following steps: using the panoramic digital scanning image to perform image processing and image segmentation to obtain a tumor center region, a tumor invasion margin region and a normal tissue region; constructing a data set by region according to the tumor center region, the tumor invasion margin region and the normal tissue region; randomly selecting 10 1000 regions through the tumor center region and the tumor invasion margin region; ×1000 The area is taken as the region of interest. The present invention uses panoramic digital scanning images for image processing and division, achieving three significant effects. First, through precise identification, the tumor center area, tumor invasion edge area and normal tissue area are successfully distinguished, providing clear and accurate pathological structure information for subsequent analysis; second, based on the division of these areas, a regional data set is constructed to ensure comprehensive data coverage and targeted analysis, which is helpful to deeply explore the pathological differences between different regions; third, by randomly selecting the tumor center and invasion edge area as the region of interest, it not only focuses on the core and edge characteristics of the tumor, but also enhances the representativeness and reliability of data analysis, providing strong data support for subsequent research, ensuring the in-depth and scientific nature of the entire analysis process, and providing an important basis for precision medicine.

[0009] Optionally, constructing the pathological genomics feature scoring model, the collagen feature scoring model and the immune feature scoring model through the region of interest includes: extracting the pathological genomics feature through the region of interest; constructing the pathological genomics feature scoring model using the regression coefficient corresponding to the pathological genomics feature; performing multiphoton imaging on the region of interest to extract collagen structural features; constructing the collagen feature scoring model using the regression coefficient corresponding to the collagen structural features; obtaining and Positive cell density; according to and Positive cell density, construct immune feature scoring model. The present invention efficiently constructs pathological genomics feature scoring model, collagen feature scoring model and immune feature scoring model through the region of interest, and achieves the following effects: first, accurately extract pathological genomics features, construct scores in combination with regression coefficients, and quantitatively evaluate tumor pathological characteristics; second, use multiphoton imaging technology to deeply extract collagen structural features, construct collagen feature scoring model, and reflect the tumor microenvironment structure; third, accurately measure and Positive cell density was used to construct an immune signature scoring model, revealing the tumor immune status.

[0010] Optionally, the pathological genomics feature scoring model satisfies the following expression: in, Scoring of pathomic features, is the number of pathological features screened, For the Pathological features, For the The regression coefficients corresponding to the pathomic features; The collagen feature scoring model satisfies the following expression: in, Scoring collagen characteristics, is the number of collagen features screened, For the Collagen characteristics, For the The regression coefficients corresponding to the collagen features; The immune signature scoring model satisfies the following expression: in, To score the immune profile, is the immune signature scoring processing function, Central area of ​​the tumor Positive cell density, Tumor invasion margin area Positive cell density, Central area of ​​the tumor Positive cell density, Tumor invasion margin area Positive cell density. The effects of the pathological genomics feature scoring model, collagen feature scoring model and immune feature scoring model constructed by the present invention are as follows: First, the pathological genomics feature scoring model comprehensively considers the pathological characteristics of tumor tissue in multiple dimensions such as morphology, structure, and function, which is conducive to obtaining comprehensive tumor pathology information; second, the collagen feature scoring model accurately considers the structural characteristics of collagen fibers in the tumor microenvironment, including their distribution, arrangement, and density, which is of great significance for evaluating the aggressiveness and prognosis of the tumor; third, the immune feature scoring model accurately predicts the treatment effect and survival outcome of patients, providing important support for the precise treatment and prognosis evaluation of colorectal cancer.

[0011] Optionally, the extraction of pathological genomic features through the region of interest includes: extracting 57 quantified image quality features, 9 image co-localization features, 48 ​​image particle features and 234 image texture features through the region of interest. The present invention extracts pathological genomic features through the region of interest, covering image quality features, image co-localization features and image texture features, and the method has achieved remarkable results. First, through the image quality features, the accuracy and reliability of the extracted information are ensured, and misjudgment caused by problems such as image blur and noise interference is avoided; second, through the image co-localization features, the spatial relationship between different pathological components is revealed, which helps to understand the complex structure and function inside the tumor; third, through the image texture features, subtle changes in tissue structure are captured, providing important clues for identifying tumor heterogeneity and evaluating tumor grading.

[0012] Optionally, performing multiphoton imaging on the region of interest to extract collagen structural features includes: performing multiphoton imaging on the region of interest to extract 142 quantified collagen structural features, including 8 morphological features and 134 texture features. The present invention has achieved remarkable results by performing multiphoton imaging on the region of interest to extract collagen structural features. First, high-resolution, three-dimensional visualization of collagen fibers is achieved, thereby accurately extracting quantified collagen structural features; second, it not only reveals the complex structure and function of collagen fibers in the tumor microenvironment, but also provides an important basis for evaluating the aggressiveness of tumors, predicting treatment responses, and formulating personalized treatment plans; third, it ensures the accuracy and reliability of the extracted features, providing strong support for the in-depth research of tumors and the development of precision medicine.

[0013] Optionally, according to the and Positive cell density, constructing an immune signature scoring model includes: and Positive cell density, obtained and The staining scores are added and converted to obtain an immune feature scoring model. and The effect of constructing an immune signature scoring model based on positive cell density is significant. and The density of positive cells can directly reflect the infiltration of T lymphocytes in the tumor tissue and at the edge of tumor invasion; second, it is calculated based on these density values. and The staining scores are used to further quantify the activity of immune cells; thirdly, the staining scores are added and converted to obtain an immune characteristic scoring model, which comprehensively reflects the intensity of tumor immune response and provides an important basis for evaluating the patient's immune status, predicting the efficacy of immunotherapy and formulating personalized immunotherapy plans.

[0014] Optionally, constructing a multimodal recurrence risk model according to the pathological genomics feature scoring model, the collagen feature scoring model and the immune feature scoring model includes: performing Cox proportional hazard regression calculation according to the pathological genomics feature scoring model, the collagen feature scoring model and the immune feature scoring model; and constructing a multimodal recurrence risk model using the calculation result, wherein the multimodal recurrence risk model satisfies the following expression: in, For the multimodal recurrence risk score, Scoring of pathomic features, is the regression coefficient corresponding to the pathological omics feature scoring model, Scoring collagen characteristics, is the regression coefficient corresponding to the collagen feature scoring model, To score the immune profile, is the regression coefficient corresponding to the immune feature scoring model. The present invention constructs a multimodal recurrence risk model based on the pathological genomics feature score, collagen feature score and immune feature score, which involves the calculation method of Cox proportional risk regression and has achieved remarkable results. First, the impact of each score on the risk of recurrence and the interaction between them are comprehensively considered to obtain a more accurate and comprehensive evaluation result; second, the prediction accuracy of the model is improved through statistical calculation methods, and a more reliable basis is provided for subsequent disease treatment and prevention; third, a multimodal recurrence risk model is used to provide a more comprehensive and objective quantitative indicator for the recurrence risk assessment of the disease, and also provides new ideas for medical research and clinical practice.

[0015] In the second aspect, the present invention provides a device for a multimodal recurrence risk model of colorectal cancer, comprising an image input device, a multimodal model processor, a recurrence risk output device and an information storage device, wherein the image input device, the multimodal model processor, the recurrence risk output device and the information storage device are interconnected, wherein the image input device comprises a pathological image module, a multiphoton image module and an immunostaining image module, the information storage device is used to store a computer program, the computer program comprises a program instruction, the multimodal model processor is configured to call the program instruction, the device executes the method for constructing a multimodal recurrence risk model of colorectal cancer, the device has a compact structure, and has significant effects in improving prediction accuracy, realizing individualized treatment and optimizing the allocation of medical resources. First, by integrating multiple modal data, more data information related to recurrence is captured, thereby improving the accuracy and reliability of prediction; second, through the prediction results of the multimodal recurrence risk model, a more individualized treatment plan is formulated for the patient; third, through the device, the patient group that needs close monitoring and active treatment is more accurately identified, thereby optimizing the allocation of medical resources, which helps to improve the efficiency and quality of medical services, and also can reduce the economic burden of patients.

[0016] The beneficial effects of the present invention compared with the prior art are as follows: based on image analysis technology, the present invention constructs a pathological genomics feature scoring model, a collagen feature scoring model and an immune feature scoring model according to digital pathological images, multiphoton images and immunohistochemical images, respectively, and integrates them into a multimodal recurrence risk model, which can be used to more comprehensively and accurately evaluate the correlation between tumors and their microenvironment and the recurrence of colorectal cancer patients; in the variable screening process, LASSO-Cox regression can effectively screen high-dimensional data, and then select the features most closely related to recurrence to construct corresponding scores; the multimodal recurrence risk model integrating the pathological genomics feature scoring model, the collagen feature scoring model and the immune feature scoring model can accurately predict the probability of recurrence in colorectal cancer patients, and its predictive efficiency is better than the traditional TNM staging system, and has important reference value in assisting individualized diagnosis and treatment decisions, improving the quality of life and improving prognosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A flowchart of a method for constructing a multimodal recurrence risk model for colorectal cancer according to an embodiment of the present invention; Figure 2 A schematic diagram of extracting pathological genomics features and constructing a pathological genomics feature scoring model according to an embodiment of the present invention; Figure 3 A schematic diagram of extracting collagen features and constructing a collagen feature scoring model according to an embodiment of the present invention; Figure 4 A schematic diagram of constructing an immune signature scoring model according to an embodiment of the present invention; Figure 5 is a nomogram of a multimodal recurrence risk model according to an embodiment of the present invention; Figure 6 Schematic diagram showing the comparison of the predictive effectiveness of the multimodal recurrence risk model according to an embodiment of the present invention and the traditional TNM staging system; Figure 7 Schematic diagram of the structure of a multimodal recurrence risk model device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0018] The specific embodiments of the present invention will be described in detail below. It should be noted that the embodiments described herein are only for illustration and are not intended to limit the present invention. In the following description, a large number of specific details are set forth in order to provide a thorough understanding of the present invention. However, it is obvious to those of ordinary skill in the art that these specific details do not need to be adopted to implement the present invention. In other examples, in order to avoid confusing the present invention, known circuits, software or methods are not specifically described.

[0019] Throughout the specification, references to "one embodiment," "an embodiment," "an example," or "an example" mean that a particular feature, structure, or characteristic described in conjunction with the embodiment or example is included in at least one embodiment of the present invention. Therefore, the phrases "in one embodiment," "in an embodiment," "an example," or "an example" appearing in various places throughout the specification do not necessarily all refer to the same embodiment or example. In addition, particular features, structures, or characteristics may be combined in one or more embodiments or examples in any suitable combination and / or subcombination. In addition, it should be understood by those of ordinary skill in the art that the figures provided herein are for illustrative purposes and that the figures are not necessarily drawn to scale.

[0020] See also Figure 1 , an embodiment of the present invention provides a method for constructing a multimodal recurrence risk model for colorectal cancer, the method comprising the following steps: S1. Obtain panoramic digital scanning images of pathological tissue sections of colorectal cancer patients.

[0021] Wherein, step S1 further comprises the following steps: S11. Obtain pathological tissue sections from patients with colorectal cancer.

[0022] In one embodiment, pathological tissue sections of colorectal cancer patients are obtained through surgical specimens.

[0023] Specifically, tumor tissue and surrounding normal tissue samples are obtained from colorectal cancer patients during surgery to ensure that the samples contain sufficient tumor tissue and invasive margins.

[0024] Furthermore, the obtained samples were immediately placed in a 10% formalin solution for fixation for no less than 6 hours to ensure the stability of cell morphology.

[0025] Furthermore, the fixed samples were dehydrated with gradient alcohol to remove water from the tissues for subsequent sectioning.

[0026] Furthermore, the dehydrated sample is immersed in molten paraffin and formed into a paraffin block after cooling, which is convenient for sectioning.

[0027] Further, the paraffin block is cut into thin slices using a microtome, typically with a thickness of 4-6 microns.

[0028] Furthermore, the sections were spread on glass slides for HE staining to obtain the first type of pathological tissue sections of colorectal cancer patients.

[0029] S12. Perform immunohistochemical staining on the first type of pathological tissue sections to obtain the second type of pathological tissue sections.

[0030] In one embodiment, using and Immunohistochemical staining of the first type of pathological tissue sections was performed using specific antibodies.

[0031] Specifically, the first type of pathological tissue sections are pre-processed, including steps such as dewaxing and hydration.

[0032] Furthermore, using and The sections were immunohistochemically stained with specific antibodies. The staining process included antigen repair, blocking endogenous peroxidase, serum blocking, adding primary antibody ( and Antibody), secondary antibody, streptavidin-peroxidase solution and DAB color development.

[0033] Furthermore, through microscopic observation, and Positive cells show specific colors, thereby distinguishing different subsets of T lymphocytes.

[0034] Furthermore, the sections are dehydrated, transparentized and sealed to finally obtain the second type of pathological tissue sections.

[0035] S13. Scan the first type of pathological tissue slices and the second type of pathological tissue slices using a scanner to obtain a panoramic digital scan image.

[0036] In one embodiment, the exclusion criteria for patient data are specified, and the first and second types of pathological tissue sections of the patient are collected.

[0037] Specifically, the first type of pathological tissue sections and the second type of pathological tissue sections are respectively placed flat on the stage of the Leica Aperio CS2 digital pathology scanner to ensure that there are no bubbles or creases.

[0038] Furthermore, the scanning program is started, and appropriate scanning parameters are selected according to the slice type, such as resolution, color depth, etc. Among them, the scanning magnification is set to 20 times.

[0039] Furthermore, the scanner captures every detail on the slice through a high-resolution lens, generating a high-quality panoramic digital image.

[0040] Furthermore, the generated digital images are saved to a computer or server for subsequent analysis, diagnosis or research.

[0041] In another embodiment, a panoramic digital scan image is obtained based on historical clinical data.

[0042] Specifically, panoramic scanning pathology images of pathological tissue sections of 658 colorectal cancer patients who underwent radical colorectal cancer surgery from January 2009 to December 2014 were screened; the panoramic scanning pathology images were obtained by using an Aperio CS2 slice scanner.

[0043] S2. Using the panoramic digital scanning image, construct a data set and select a region of interest.

[0044] Wherein, step S2 further comprises the following steps: S21. Perform image processing and image segmentation using the panoramic digital scanning image to obtain a tumor center area, a tumor invasion edge area, and a normal tissue area.

[0045] In one embodiment, the panoramic digital scan image is used to extract the boundary line between the tumor area and the normal tissue area, and the area containing the tumor is determined as the initial tumor center area, and the area containing the normal tissue is determined as the initial normal tissue area.

[0046] Furthermore, the boundary line is translated toward the direction of the initial tumor center region by a first distance to form a first boundary line, and the boundary line is translated toward the direction of the initial normal tissue region by a second distance to form a second boundary line.

[0047] Further, the area between the first boundary line and the second boundary line is determined as the tumor invasion margin area.

[0048] Further, the area in the initial tumor center area except the invasion edge area is determined as the tumor center area, and the area in the initial normal tissue area except the invasion edge area is determined as the normal tissue area.

[0049] S22. Constructing a data set by region according to the tumor center region, the tumor invasion edge region and the normal tissue region.

[0050] The data set includes, but is not limited to, data on quality features, co-localization features, particle features, texture features, and collagen structure features of the image.

[0051] In one embodiment, feature data sets of the tumor center region, the tumor invasion edge region, and the normal tissue region are constructed separately, and then the feature data sets are integrated to form a complete data set.

[0052] Specifically, a combination of automation and semi-automation is used to extract feature data to ensure efficiency and accuracy. Automated methods such as deep learning algorithms can quickly extract a large number of features, while semi-automated methods allow for fine-tuning and verification of key features.

[0053] Furthermore, the extracted feature data is associated with the corresponding region labels to construct a complete data set, which contains not only the original image data but also rich feature data and label information, providing strong support for subsequent data analysis and model training.

[0054] S23. Randomly select 10 1000 ×1000 The area is taken as the region of interest.

[0055] In one embodiment, the tumor invasion margin and tumor center are selected using Aperio Image Scope (12.3.3) software. After determining the tumor invasion margin area and tumor center area, 10 1000 pixels are randomly selected from each of the tumor invasion margin area and tumor center area. ×1000 The area was taken as the region of interest for subsequent analysis.

[0056] S3. Using the region of interest, a pathological genomics feature scoring model, a collagen feature scoring model and an immune feature scoring model are constructed.

[0057] Wherein, step S3 further comprises the following steps: S31. Construct a pathological genomics feature scoring model based on the region of interest.

[0058] Specifically, step S31 includes the following steps: S311. Extract pathological genomic features through the region of interest.

[0059] In one embodiment, the pathological features are extracted using CellProfiler (4.1.3) software, such as Figure 2 shown.

[0060] Specifically, 348 pathological features were extracted from each region of interest, including 57 image quality features, 9 image colocalization features, 48 ​​image granularity features and 234 image texture features.

[0061] In this embodiment, after removing plaques with a background greater than 50%, the "UnmixColors" module is used to separate the region of interest into grayscale images of hematoxylin staining and eosin staining; then the "Combine" module and the "ColorToGray" module are used to convert the digital HE staining image into a grayscale image for subsequent analysis.

[0062] Specifically, the “Measure Image Quality” module is used to extract image quality features of hematoxylin, eosin and HE images, respectively. The image quality features include integrated intensity, mean intensity, standard deviation of intensity, minimum intensity, maximum intensity, integrated intensity edge, mean intensity edge, standard deviation of intensity edge, minimum intensity edge, maximum intensity edge, mass displacement, lower quartile intensity, median intensity, median absolute deviation intensity, upper quartile intensity, center mass intensity_X, center mass intensity_Y, maximum intensity_X, and center mass intensity_Y. Intensity_X), Maximum Intensity_Y (MaxIntensity_Y).

[0063] Furthermore, the “Measure Colocalization” module was used to extract the colocalization features of the HE image (Imagecolocalization features), including Correlation_Eosin_Haematoxylin, Costes Autothreshold_Eosin, Costes Autothreshold_Eosin, Manders Coefficient_Eosin_Haematoxylin, Manders Coefficient_Eosin, Overlap_Eosin_Haematoxylin, RWC_Eosin_Haematoxylin, RWC_Haematoxylin_Eosin, and Slope_Eosin_Haematoxylin.

[0064] Furthermore, HE, hematoxylin and eosin image granularity features were extracted using the “Measure Granularity” module, which returns a measurement value for each instance set within the granularity spectrum, and ultimately 48 granularity features were extracted.

[0065] Furthermore, the “Measure Texture” module was used to extract the median variance (Median Variance), standard deviation of variation (Std dev Variance), minimum inverse difference moment (MinInverseDifferenceMoment), maximum inverse difference moment (Max InverseDifferenceMoment), mean inverse difference moment (Mean InverseDifferenceMoment), median inverse difference moment (MedianInverseDifferenceMoment), standard deviation of inverse difference moment (Std dev InverseDifferenceMoment), minimum sum average (Min SumAverage), maximum sum average (Max SumAverage), mean sum average (MeanSumAverage), median of the mean (Median SumAverage), standard deviation of the mean (Std devSumAverage), minimum sum variance value (Min SumVariance), maximum sum variance value (Max SumVariance), mean sum variance value (Mean SumVariance), and median variance value (Median SumVariance), and standard deviation of variance (Std dev SumVariance), minimum sum entropy (Min SumEntropy), maximum sum entropy (MaxSumEntropy), mean sum entropy (Mean SumEntropy), median sum entropy (Median SumEntropy), standard deviation of sum entropy (Std dev SumEntropy), minimum entropy (Min Entropy), maximum entropy (MaxEntropy), mean entropy (Mean Entropy), median entropy (Median Entropy), standard deviation of entropy (Std devEntropy), minimum variance difference (Min DifferenceVariance), maximum variance difference (MaxDifferenceVariance), mean variance difference (Mean DifferenceVariance), median variance difference (Median DifferenceVariance), standard deviation of variance difference (Std dev DifferenceVariance), minimum entropy difference (Min DifferenceEntropy), maximum entropy difference (Max DifferenceEntropy),Mean entropy difference (MeanDifferenceEntropy), median entropy difference (Median DifferenceEntropy), standard deviation of entropy difference (Stddev DifferenceEntropy), minimum repeated area information 1 (Min InfoMeas1), maximum repeated area information 1 (MaxInfoMeas1), mean repeated area information 1 (Mean InfoMeas1), median repeated area information 1 (MedianInfoMeas1), standard deviation of repeated area information 1 (Std dev InfoMeas1), minimum repeated area information 2 (MinInfoMeas2), maximum repeated area information 2 (Max InfoMeas2), mean repeated area information 2 (MeanInfoMeas2), median repeated area information 2 (Median InfoMeas2), standard deviation of repeated area information 2 (Std devInfoMeas2). ,

[0066] S312. Using the regression coefficients corresponding to the pathological genomics features, a pathological genomics feature scoring model is constructed.

[0067] Since the extracted high-dimensional pathological omics features contain a lot of redundant information, and too many features can easily lead to overfitting and reduce the generalization ability of the model, it is necessary to use machine learning methods to select features and reduce data dimensionality to make the model more accurate and concise, such as Figure 2 As shown in the figure, the horizontal axis represents the penalty coefficient, recorded as log (λ), and the vertical axis Coefficients represents the regression coefficient. The pathological omics features most closely related to prognosis are screened out through LASSO-Cox regression, and their regression coefficients are reduced to a reasonable range, and a pathological omics feature scoring model is constructed.

[0068] Specifically, in one embodiment, the extracted pathological omics features are combined with the patient's prognostic information to form a data set.

[0069] Furthermore, the LASSO-Cox regression model was used to perform multivariate analysis of the data to evaluate the impact of each pathological genomic feature on patient prognosis; the regression coefficient corresponding to each feature was determined, which reflects the contribution of the corresponding feature to the patient's prognostic risk.

[0070] Furthermore, a pathology feature scoring model is constructed according to the regression coefficient, the pathology feature and the number of screened pathology features. The pathology feature scoring model satisfies the following expression: in, Scoring of pathomic features, is the number of pathological features screened, For the Pathological features, For the The regression coefficients corresponding to the pathomic features.

[0071] In this embodiment, =13; in, Represents the comprehensive strength of hematoxylin, is 0.4271; in, represents the minimum intensity of hematoxylin, is 204.3945; in, Represents the maximum intensity of hematoxylin, is -112.3398; in, Representative of the characteristics of eosin granules 3, is 0.3423; in, Represents the hematoxylin granule intensity feature 13, is 0.0023; in, Represents HE particle strength characteristic 3, is 0.4999; in, Represents HE particle strength characteristic 13, is 0.1257; in, represents the mean variance of eosin, is 0.005; in, represents the hematoxylin angular second moment, is 0.0785; in, represents the mean hematoxylin contrast, is 0.0598; in, represents the maximum correlation coefficient of hematoxylin, is -42.1865; in, represents the minimum variance of hematoxylin, is 6113.2450; in, represents the sum of HE standard deviations, It is 0.0028.

[0072] S32. Perform multiphoton imaging on the region of interest and construct a collagen feature scoring model.

[0073] Specifically, step S32 includes the following steps: S321. Perform multiphoton imaging on the region of interest to extract collagen structural features.

[0074] In one embodiment, multiphoton imaging of the region of interest is performed using a Zeiss LSM 880, such as Figure 3 As shown. Among them, the imaging magnification is 20 times, the excitation wavelength is 810nm, and the 32-channel GaAsP photomultiplier tube array detector is used to collect the second harmonic signal (green) with a wavelength range of 387nm~409nm and the two-photon excitation fluorescence (red) signal with a wavelength range of 430nm~708nm, and the multiphoton imaging image is compared with the HE image. Matlab (2023b) software was used to extract 142 quantitative collagen structural features from the multiphoton imaging image, including 8 morphological features and 134 texture features.

[0075] Among them, the collagen morphological characteristic parameters include: collagen area, collagen number, collagen length, collagen fiber width, collagen straightness, collagen cross-link density, collagen cross-link space and collagen orientation.

[0076] Among them, collagen texture features include histogram mean, histogram variance, histogram skewness, histogram kurtosis, histogram energy and histogram entropy; 80 gray-level co-occurrence matrices (GLCM): the above 80 GLCMs are obtained by calculating the contrast, correlation, energy and uniformity of 5 pixels at positions 1, 2, 3, 4 and 5 and 4 different directions of 0°, 45°, 90° and 135°; 48 Gabor wavelet transform parameters: the means and variances of 4 different scales and 5 different directions are used to obtain 48 Gabor wavelet transform parameters.

[0077] S322. Construct a collagen characteristic scoring model using the regression coefficients corresponding to the collagen structural characteristics.

[0078] Since the extracted high-dimensional collagen features contain a lot of redundant information, and too many features can easily lead to overfitting and reduce the generalization ability of the model, it is necessary to use machine learning methods to select features and reduce data dimensionality to make the model more accurate and concise. Please refer to Figure 3 The horizontal axis represents the penalty coefficient, recorded as log (λ), and the vertical axis Coefficients represents the regression coefficient. The collagen characteristics most closely related to prognosis were screened out through LASSO-Cox regression, and their regression coefficients were reduced to a reasonable range, and a collagen characteristic scoring model was constructed.

[0079] Specifically, in one embodiment, the extracted collagen features are combined with the patient's prognostic information to form a data set.

[0080] Furthermore, the LASSO-Cox regression model was used to perform multivariate analysis of the data to evaluate the impact of each collagen feature on patient prognosis; the regression coefficients corresponding to each feature were determined, which reflected the contribution of the collagen feature to the patient's prognostic risk.

[0081] Furthermore, a collagen feature scoring model is constructed based on the regression coefficient, the collagen feature and the number of screened collagen features. The collagen feature scoring model satisfies the following expression: in, Scoring collagen characteristics, is the number of collagen features screened, For the Collagen characteristics, For the The regression coefficients corresponding to the collagen features.

[0082] In this embodiment, =11; in, represents the collagen area, is 2.5936; in, represents the histogram skewness, is -0.0521; in, represents the bar graph energy, is 0.9799; in, Represents the correlation of the pixel distance 4 in the 0° direction in the gray-level co-occurrence matrix, is -3.9711; in, Represents the contrast of the pixel distance 5 in the 0° direction in the gray-level co-occurrence matrix, is -8.1273; in, Represents the correlation of the pixel distance 3 in the 45° direction in the gray-level co-occurrence matrix, is -1.323; in, Represents the correlation of the pixel distance 1 in the 135° direction in the gray-level co-occurrence matrix. is 7.7518; in, Represents the correlation of the pixel distance 5 in the 135° direction in the gray-level co-occurrence matrix, is -4.4911; in, represents the variance of the Gabor filter image convolution with a scale of 1 and a directionality of 30°, is 00.1153; in, represents the variance of the Gabor filter image convolution with a scale of 1 and a direction of 180°, is -6.6137; in, represents the variance of the Gabor filter image convolution with a scale of 2 and a directionality of 60°, is 4.3175.

[0083] S33. Construct an immune signature scoring model based on the region of interest.

[0084] Specifically, step S33 includes the following steps: S331. Obtaining the region of interest and Positive cell density.

[0085] In one embodiment, and The antibody was used to perform immunohistochemical staining on the area of ​​interest in the pathological tissue sections to obtain and Positive cell density, such as Figure 4 The digital images of the stained sections were stored as digital images at 20 times magnification by a Leica AperioCS2 digital pathology scanner.

[0086] Specifically, the image analysis software QuPath was used to segment and identify cells within the region of interest, which involved steps such as color threshold setting, morphological filtering, and cell outline detection.

[0087] Furthermore, for and Staining, set appropriate color channels and thresholds to distinguish positive cells from negative cells.

[0088] Furthermore, in the segmented image, the and The number of positive cells. This is achieved by counting labeled cells in the software or by analyzing the generated cell list.

[0089] Further, calculate the density of positive cells (i.e., number of cells / area) based on the area of ​​the region of interest, making sure to use the same units (e.g., cells / mm2) for comparison.

[0090] It should be noted that QuPath is an open source and free pathology image analysis platform that can automatically count positive cells in annotated images and group them according to the positive cell density of each image. Patients with a density ≥75% are considered to be in the "high immunity group", and vice versa.

[0091] S332. According to and Positive cell density was used to construct an immune signature scoring model.

[0092] In one embodiment, according to and Positive cell density, an immune signature scoring model was constructed, and the immune signature scoring model satisfies the following expression: in, To score the immune profile, is the immune signature scoring processing function, Central area of ​​the tumor Positive cell density, Tumor invasion margin area Positive cell density, Central area of ​​the tumor Positive cell density, Tumor invasion margin area Positive cell density.

[0093] It should be noted that The function is to , , and The staining scores are added and converted into immune scores, which are divided into five levels, corresponding to I0, I1, I2, I3, and I4; the immune scores are then divided into three groups: I0-I1 is a low immune score, I2 is a medium immune score, and I3-I4 is a high immune score.

[0094] S4. Construct a multimodal recurrence risk model based on the pathological genomics feature scoring model, the collagen feature scoring model and the immune feature scoring model.

[0095] In one embodiment, the Cox proportional hazard regression is calculated using the pathological genomics feature scoring model, the collagen feature scoring model, and the immune feature scoring model, and a multimodal recurrence risk model is constructed based on the calculated results; the multimodal recurrence risk model satisfies the following expression: in, To develop a multimodal recurrence risk score, Scoring of pathomic features, is the regression coefficient corresponding to the pathological omics feature scoring model, Score collagen characteristics, is the regression coefficient corresponding to the collagen feature scoring model, To score the immune profile, is the regression coefficient corresponding to the immune characteristic scoring model. The Cox proportional hazard regression method was used to calculate the is 1.101, is 0.388, It is -0.829.

[0096] See also Figure 5 , Figure 5The multimodal recurrence risk model nomogram disclosed in the present invention is as follows: The multimodal recurrence risk model is explained by the patient's historical data.

[0097] (1) The calculated pathological omics score for patient A was -0.157, the collagen score was -0.518, and the immune score was 1.00. The multimodal recurrence risk score of this patient was -1.203, and the corresponding score point of the nomogram was 26 points. The probability of disease-free survival for this patient at 2 years was 92%, the probability of disease-free survival for 3 years was 91%, and the probability of disease-free survival for 5 years was 82%. The risk of recurrence was low, and the frequency of follow-up could be reduced.

[0098] (2) The calculated pathological omics feature score for patient B was 1.126, the collagen feature score was 1.312, and the immune feature score was 0.00. The multimodal recurrence risk score of this patient was 1.749, and the corresponding score point of the nomogram was 78 points. The probability of disease-free survival for this patient at 2 years was 28%, the probability of disease-free survival for 3 years was 18%, and the probability of disease-free survival for 5 years was <10%. The risk of recurrence was high, and follow-up after surgery was required, and timely treatment was required if recurrence was found.

[0099] See also Figure 6 , Figure 6 It is a schematic diagram of the time-dependent receiver operating characteristic curve (ROC) of the multimodal recurrence risk model disclosed in the present invention and the TNM staging system for predicting recurrence. The area under the curve (AUC) is defined as the area enclosed by the coordinate axis under the ROC curve, and the value range is 0.5 to 1. The closer the AUC is to 1, the higher the prediction model performance. The AUC value of the 5-year DFS of the prediction model constructed by the present invention is 0.900, and the AUC value of the 5-year DFS predicted by the TNM staging system is 0.746. It can be seen from this that the predictive efficiency of the multimodal recurrence risk model disclosed in the present invention is significantly higher than that of the TNM staging system.

[0100] See also Figure 7 , Figure 7The present invention is a schematic diagram of the structure of a multimodal recurrence risk device for colorectal cancer according to an embodiment of the present invention. The device includes an image input device 710, a multimodal model processor 720, a recurrence risk output device 730 and an information storage device 740, wherein the image input device 710, the multimodal model processor 720, the recurrence risk output device 730 and the information storage device 740 are interconnected, wherein the image input device 710 includes a pathological image module 711, a multiphoton image module 712 and an immunostaining image module 713, the information storage device 740 is used to store a computer program, the computer program includes program instructions, the multimodal model processor 720 is configured to call the program instructions, and the device executes the method for constructing a multimodal recurrence risk model for colorectal cancer.

[0101] In this embodiment, the image input device 710 has a built-in pathological image module 711, a multiphoton image module 712 and an immunostaining image module 713, and is used to obtain pathological images, multiphoton images and immunostaining images.

[0102] The multimodal model processor 720 serves as the core of the device, including one or more central processing units (CPU), with built-in model algorithms, including pathological genomics feature scoring model, collagen feature scoring model, immune feature scoring model and multimodal recurrence risk model algorithms; the model is mainly trained by using historical image data of patient pathological tissue sections to obtain an optimized model, and further combined with the image of the current pathological tissue section to obtain the pathological tissue score, collagen feature score, immune feature score and multimodal recurrence risk score.

[0103] The recurrence risk output device 730 includes a display screen, which can output the obtained recurrence risk to the target terminal, and is used to display the calculation results of the pathological tissue feature score, the collagen feature score model, the immune feature score model and the multimodal recurrence risk score, as well as the prediction results of the patient's colorectal cancer multimodal recurrence risk based on the calculation results.

[0104] The information storage 740 includes local storage and cloud storage, which can provide instructions and data to the multimodal model processor 720. It is mainly used to store data input by the image input device 710 and data obtained by processing by the multimodal model processor 720, and can meet the needs of large data storage; the information storage includes a high-speed solid-state hard drive, which has the characteristics of fast reading and writing speed, large capacity and high reliability.

[0105] The multimodal recurrence risk model device further includes one or more power supplies 750, one or more operating systems 760, such as Windows Server, Mac OSX, Unix, Linux, FreeBSD, and one or more wired or wireless network interfaces 770. It will be appreciated by those skilled in the art that the computer device structure does not constitute a limitation on the computer device provided by the present invention, and may include, for example, Figure 7 More or fewer components, or some components in combination, or a different arrangement of components.

[0106] In summary, the method for constructing a multimodal recurrence risk model for colorectal cancer provided by the present invention is based on the pathological tissue sections of colorectal cancer patients and provides an innovative means for predicting the risk of disease recurrence through a series of fine steps. Through panoramic digital scanning images, the comprehensiveness and accuracy of pathological information are ensured, laying a solid foundation for subsequent analysis; by constructing data sets and outlining regions of interest, the efficiency and pertinence of data processing are effectively improved; by constructing a pathological genomics feature scoring model, a collagen feature scoring model, and an immune feature scoring model, the pathological characteristics are reflected in multiple dimensions, providing a scientific basis for accurately predicting the probability of colorectal cancer recurrence; by integrating pathological genomics, collagen features, and immunological indicators to construct a multimodal recurrence risk model, it not only comprehensively considers a variety of pathophysiological factors, but also significantly improves the accuracy and reliability of recurrence risk prediction, showing broad application prospects in the field of colorectal cancer recurrence risk assessment. The device for the multimodal recurrence risk model of colorectal cancer provided by the present invention captures more data information related to recurrence by integrating multiple modal data, thereby improving the accuracy and reliability of prediction; through the prediction results of the multimodal recurrence risk model, a more individualized treatment plan can be formulated for the patient; through this device, the patient group that needs close monitoring and active treatment can be more accurately identified, thereby optimizing the allocation of medical resources, helping to improve the efficiency and quality of medical services, and also reducing the economic burden on patients.

[0107] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and specification of the present invention.

Claims

1. A method for constructing a multimodal recurrence risk model for colorectal cancer, characterized in that: The construction method comprises the following steps: Obtain panoramic digital scanning images of pathological tissue sections of patients with colorectal cancer; Using the panoramic digital scanned image, constructing a data set and selecting an area of ​​interest; Using the region of interest, constructing a pathological omics feature scoring model, a collagen feature scoring model, and an immune feature scoring model; A multimodal recurrence risk model is constructed based on the pathological omics feature scoring model, the collagen feature scoring model and the immune feature scoring model.

2. The method for constructing a multimodal recurrence risk model for colorectal cancer according to claim 1, characterized in that: The method of obtaining a panoramic digital scanning image of a pathological tissue section of a colorectal cancer patient comprises the following steps: Obtain the first type of pathological tissue sections from patients with colorectal cancer; Performing immunohistochemical staining on the first type of pathological tissue sections to obtain second type of pathological tissue sections; The first type of pathological tissue slices and the second type of pathological tissue slices are scanned by using a scanner to obtain a panoramic digital scan image.

3. The method for constructing a multimodal recurrence risk model for colorectal cancer according to claim 1, characterized in that: The method of constructing a data set and selecting a region of interest using the panoramic digital scanned image comprises the following steps: Using the panoramic digital scanning image to perform image processing and image segmentation, obtaining a tumor center area, a tumor invasion edge area, and a normal tissue area; Constructing a data set by region according to the tumor center region, the tumor invasion edge region and the normal tissue region; Ten 1000 ×1000 The area is taken as the region of interest.

4. The method for constructing a multimodal recurrence risk model for colorectal cancer according to claim 1, characterized in that: The constructing of the pathological genomics feature scoring model, the collagen feature scoring model and the immune feature scoring model through the region of interest includes: Extracting pathological genomic features through the region of interest; constructing a pathological genomic feature scoring model using the regression coefficients corresponding to the pathological genomic features; Performing multiphoton imaging on the region of interest to extract collagen structural features; constructing a collagen feature scoring model using regression coefficients corresponding to the collagen structural features; Through the region of interest, we obtain and Positive cell density; according to and Positive cell density was used to construct an immune signature scoring model.

5. The method for constructing a multimodal recurrence risk model for colorectal cancer according to claim 4, characterized in that: The pathological genomics feature scoring model satisfies the following expression: in, Scoring of pathomic features, is the number of pathological features screened, For the Pathological features, For the The regression coefficients corresponding to the pathomic features; The collagen feature scoring model satisfies the following expression: in, Scoring collagen characteristics, is the number of collagen features screened, For the Collagen characteristics, For the The regression coefficients corresponding to the collagen features; The immune signature scoring model satisfies the following expression: in, To score the immune profile, is the immune signature scoring processing function, Central area of ​​the tumor Positive cell density, Tumor invasion margin area Positive cell density, Central area of ​​the tumor Positive cell density, Tumor invasion margin area Positive cell density.

6. The method for constructing a multimodal recurrence risk model for colorectal cancer according to claim 4, characterized in that: According to the and Positive cell density, construction of immune signature scoring model includes: According to the and Positive cell density, obtained and The staining score; The staining scores are added and transformed to obtain an immune signature scoring model.

7. The method for constructing a multimodal recurrence risk model for colorectal cancer according to claim 4, characterized in that: The extracting of pathological omics features through the region of interest includes: Through the region of interest, 57 quantitative image quality features, 9 image co-localization features, 48 ​​image particle features and 234 image texture features are extracted.

8. The method for constructing a multimodal recurrence risk model for colorectal cancer according to claim 4, characterized in that: The performing multiphoton imaging on the region of interest to extract collagen structural features comprises: Multiphoton imaging is performed on the region of interest to extract 142 quantitative collagen structural features, which include 8 morphological features and 134 texture features.

9. The method for constructing a multimodal recurrence risk model for colorectal cancer according to claim 1, characterized in that: The constructing of a multimodal recurrence risk model according to the pathological omics feature scoring model, the collagen feature scoring model and the immune feature scoring model comprises: Performing Cox proportional hazard regression calculations according to the pathological omics feature scoring model, the collagen feature scoring model, and the immune feature scoring model; Using the calculated results, a multimodal recurrence risk model is constructed, and the multimodal recurrence risk model satisfies the following expression: in, To develop a multimodal recurrence risk score, Scoring of pathomic features, is the regression coefficient corresponding to the pathological omics feature scoring model, Score collagen characteristics, is the regression coefficient corresponding to the collagen feature scoring model, To score the immune profile, is the regression coefficient corresponding to the immune signature scoring model.

10. A device for a multimodal recurrence risk model for colorectal cancer, the device executing a method for constructing a multimodal recurrence risk model for colorectal cancer according to any one of claims 1 to 9, characterized in that: The apparatus comprises an image input device, a multimodal model processor, a recurrence risk output device and an information storage device, wherein the image input device, the multimodal model processor, the recurrence risk output device and the information storage device are interconnected, wherein the image input device comprises a pathological image module, a multiphoton image module and an immunostaining image module, the information storage device is used to store a computer program, the computer program comprises program instructions, and the multimodal model processor is configured to call the program instructions.

Images