Middle-low rectal cancer out-of-wall invasion degree automatic sketching method based on multi-mode MRI (Magnetic Resonance Imaging)

By developing a method combining multimodal MRI image matching with AI algorithm in medium and low rectal cancer, the accuracy of the assessment of the degree of extra-wall invasion of medium and low rectal cancer is solved, and accurate diagnosis and personalized treatment plans are achieved, which improves the Efficiency and effectiveness of clinical diagnosis and treatment.

CN118967849APending Publication Date: 2024-11-15TIANJIN PEOPLE HOSPITAL
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202410932290.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-12
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

The prior art is difficult to accurately evaluate the degree of extramural invasion of medium and low rectal cancer, resulting in poor surgical regimen selection and treatment effectiveness, especially in the absence of intraoperative frozen conditions.

Method used

Develop a method for outlined the degree of extra-wall invasion of medium and low rectal cancer based on multimodal MRI, realize the matching and fusion of T2WI and DWI images through unsupervised network, combine AI algorithm to outline the target area to establish the degree of extra-wall invasion of medium and low rectal cancer. Database to improve diagnostic accuracy and efficiency.

Benefits of technology

It has achieved accurate assessment of the degree of extra-wall invasion of medium and low rectal cancer, reduced manual outline error and time cost, improved clinical work efficiency, and provided a basis for personalized treatment plans, improving treatment effect and prognosis.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure BSA0000299593900000041
    Figure BSA0000299593900000041
  • Figure HSA0000299593910000011
    Figure HSA0000299593910000011
  • Figure HSA0000299593910000012
    Figure HSA0000299593910000012
Patent Text Reader

Abstract

The invention provides a method for automatically sketching out-of-wall invasion degree of middle and low rectal cancer based on multi-modal MRI, and develops an unsupervised network for simultaneously realizing feature point detection and descriptor generation on a full-size image for the problem of T2WI and DWI image matching in middle and low rectal cancer MRI, so that fusion of T2WI and DWI images can be realized; on the basis, by expanding a known data set, a target region sketching network of the extracorporeal invasion degree of the mid-low rectal cancer is constructed, and a basis of a personalized treatment scheme is provided for clinic.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image processing, and in particular relates to a method for outlining the degree of extramural invasion of mid- and low-level rectal cancer based on multimodal MRI. Background Art

[0002] Colorectal cancer (CRC) is the third most common cancer in the world and the second leading cause of cancer death. It is a common malignant tumor in the digestive system with a poor prognosis in the late stage. Among CRC, the incidence of rectal cancer is higher than that of colon cancer. Statistical analysis at home and abroad shows that in 2020, rectal cancer became the eighth most common cancer in the world with more than 730,000 cases, causing nearly 340,000 deaths. Although the mortality rate of early CRC has decreased with the increase in people's enthusiasm for active participation in national colorectal cancer screening and the continuous improvement of rectal cancer treatment methods, the overall survival rate of CRC is still low because late-stage CRC is prone to recurrence and metastasis and the treatment effect is poor, posing a serious threat to human life and health.

[0003] In the past few decades, with the development of modern high-tech and the improvement of treatment strategies, the treatment ideas and strategies of rectal cancer have undergone tremendous changes. For example, the introduction of the standard treatment of total mesorectal excision (TME) combined with neoadjuvant chemoradiotherapy and minimally invasive surgery such as laparoscopy has broadened the treatment options for rectal cancer, but radical surgical resection is still the main effective treatment for patients with rectal cancer. The Chinese Guidelines for the Diagnosis and Treatment of Colorectal Cancer (2023 Edition) proposes that the resection of middle and lower rectal cancer must follow the TME principle and free the mesorectum as sharply as possible. On the premise of radical treatment of the tumor, the function of the anal sphincter, urination and sexual function should be preserved as much as possible. Due to the complex pelvic anatomical structure and rich periintestinal blood vessels and nerves, surgical treatment of middle and low rectal cancer is very challenging. The rectum is connected to the anal canal and anus. If the tumor is close to the anal canal, the surgery may damage its function, and even the anal canal and anus must be removed, resulting in the patient's inability to defecate normally, suffering great physical and mental trauma, and seriously affecting the patient's quality of life. Sphincter-preserving surgery is aimed at patients with low or ultra-low rectal cancer. It requires accurate preoperative diagnosis and evaluation, reasonable surgical method selection, standardized perioperative management and rehabilitation exercises, which are the key to ensuring the radical cure of tumors and functional preservation in sphincter-preserving surgery. In addition, negative surgical margins should be ensured. Because some regions and hospitals do not have intraoperative freezing conditions, it is impossible to quickly judge the circumferential resection margin and the degree of invasion of the rectal cancer wall, which makes it difficult to evaluate the preoperative evaluation and choose the surgical plan.

[0004] Auxiliary examinations for rectal cancer include rectal digital examination, transrectal ultrasound, electronic colonoscopy, pelvic CT, pelvic MRI, and 18F-positron emission tomography (PET / CT). As one of the important auxiliary examinations for rectal cancer, imaging examinations can help clinicians more accurately conduct TNM staging assessments for rectal tumors, thereby stratifying and individualizing patients for treatment. Among them, the advantages of pelvic MRI are more prominent. This examination method not only has high soft tissue resolution and diversified image post-processing technology, but also has the function of multi-directional imaging. These advantages enable pelvic MRI examinations to clearly identify various key anatomical structures and clearly observe the degree of tumor invasion. Therefore, pelvic MRI is currently the preferred imaging method for rectal cancer. Multimodal MRI imaging is a way to integrate multiple sequence images. Imaging physicians can obtain more information through multimodal images. Multimodal MRI technology can analyze the mechanism of disease occurrence and development from multiple angles. The integration between different modalities is an important way to understand the correlation between pelvic structure and functional changes.

[0005] Studies have shown that the positive predictive value of MRI for predicting CRM is low, suggesting overdiagnosis of CRM invasion. However, the hazard ratio of local recurrence based on the relevant CRM in the pathological specimen is almost twice that of the hazard ratio of local recurrence based on MRI for predicting CRM (8.80 vs. 4.25), which also suggests overdiagnosis of CRM invasion. At present, the EMVI (magnetic resonance imaging-defined EMVI, mrEMVI) score based on high-resolution MRI is the main method for preoperative evaluation of EMVI status and an independent risk factor for tumor staging and liver metastasis. However, previous research results have shown that compared with the gold standard of postoperative pathological examination, the sensitivity of the mrEMVI score is low and the difference is large (28.2%-62.0%). Due to the limitations of clinicians' image processing technology, there has been no breakthrough in the accurate estimation of CRM and EMVI using MRI in recent years. Predicting EMVI and CRM by accurately evaluating the degree of extramural invasion of rectal cancer is an urgent problem to be solved at the technical level. Effective and reliable EMVI and CRM evaluation can help clinicians formulate precise treatment plans and improve treatment efficacy. When formulating surgical plans for patients with low rectal cancer and ultra-low rectal cancer, whether the anal preservation situation can be accurately judged is of great significance to the patient's quality of life. Summary of the invention

[0006] Objective: This paper is based on the multimodal pelvic MRI database of middle and low rectal cancer, aiming to accurately identify and evaluate the extramural invasion of tumor lesions. In view of the current lack of automatic delineation technology for pelvic tumors in clinical practice, with the goal of realizing intelligent auxiliary analysis of the degree of extramural invasion of middle and low rectal cancer, multimodal image MRI fusion is performed to mine the detailed structural information in MRI images; and based on this, AI algorithms are introduced to complete the rapid measurement and accurate identification of the distance between the extramural invasion edge of the tumor and the anal verge, and realize pixel-level delineation and image information evaluation of rectal tumor MRI images, which can not only provide a basis for the clinical formulation of appropriate personalized treatment plans as soon as possible, but also provide effective evaluation of the treatment effect and prognosis of patients.

[0007] Design: The present invention relates to a method for outlining the extent of extramural invasion of mid- and low-level rectal cancer based on multimodal MRI, comprising the following steps:

[0008] (1) An unsupervised network was developed to simultaneously detect feature points and generate descriptors on full-size images for matching and fusion of T2WI and DWI images in MRI of mid- and low-level rectal cancer.

[0009] (2) Develop a miniaturized method for constructing a database of the extent of extramural invasion of mid- and low-lying rectal cancer;

[0010] (3) Design a target volume delineation network for the degree of extramural invasion of mid- and low-level rectal cancer.

[0011] Beneficial effects: The method for outlining the extent of extramural invasion of mid- and low-level rectal cancer based on multimodal MRI provided by the present invention has the following advantages:

[0012] ① The present invention achieves accurate assessment of the degree of extramural invasion of mid- and low-level rectal cancer by combining the fusion of multimodal MRI images with AI algorithms, thus improving the accuracy and efficiency of diagnosis;

[0013] ② Through automated processing, the errors and time costs caused by manual delineation are reduced, and clinical work efficiency is improved;

[0014] ③This method can be deployed in the actual hospital environment and optimized based on doctors' feedback to ensure that the system operates stably and meets clinical needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a neural network structure diagram outlining the degree of extramural invasion of mid- and low-position rectal cancer in multimodal MRI provided by the present invention.

[0016] Figure 2 It is a schematic diagram of the delineation results of the degree of extramural invasion of mid- and low-position rectal cancer by multimodal MRI provided by the present invention. DETAILED DESCRIPTION

[0017] Now combined Figure 1 The method for outlining the degree of extramural invasion of mid-low rectal cancer in multimodal MRI provided by the present invention is described in detail. It should be understood that the drawings are only exemplary and do not limit the scope of the present invention.

[0018] like Figure 1 As shown, the present invention provides a working method of a method for outlining the degree of extramural invasion of mid- and low-level rectal cancer using multimodal MRI, comprising the following steps:

[0019] ① This method develops an image registration and data extraction neural network model for the features of MRI images of mid- and low-level rectal cancer, which is used to match T2WI and DWI images. The feature of this model is that it can simultaneously realize the unsupervised network of feature point detection and descriptor generation on the full-size image. This method can extract pixel-level feature points. In real scenes with large field of view changes, the feature point detection ability of this method is better than the traditional manual design method. The model structure of the network is mainly divided into the encoding part and the decoding part. In the encoding part, the input is a grayscale image with a batch of 1, which is mainly composed of convolutional layers, Max-Pooling layers, and nonlinear activation layers. After 8 convolution operations and three Max-Pooling layers, the image size is changed to 1 / 8 of the original size, and the image changes from W×H×1 to W / 8×H / 8×256. This paper mainly reparameterizes this part. For the decoding part, there are two branches: feature point extraction branch and descriptor generation branch. The feature point extraction branch outputs the probability information that each pixel in the original image is a feature point, with a value between 0 and 1, while the descriptor generation branch outputs the descriptor corresponding to each pixel, and each descriptor consists of a 256-dimensional floating-point feature vector. The feature point detection branch of the decoding part first performs two convolution operations to obtain the parameters of 64 channels, and then adds a non-feature point channel to prevent the abnormal randomness of the response on each channel after softmax due to noise interference, and avoid judging non-feature points as feature points in the subsequent feature point selection step. By doing softmax on the channel dimension, then removing the dustbin channel, and then shuffling the pixels along the 64 channels, an 8×8 cell block can be obtained for every 64 pixels. The cell blocks are spliced ​​together to form a feature point probability matrix with the same size as the input image. The value of each pixel corresponds to the probability value of the pixel coordinate of the input image being a feature point. The larger the probability value, the higher the possibility that the corresponding position of the input image is a feature point. After that, the non-maximum suppression and edge suppression methods can be used to remove the dense points of repeated detection and the erroneous detection points at the edge position, and then the points with higher probability are screened out by the threshold to obtain the feature point coordinates with uniform distribution and pixel-level positioning. The descriptor generation branch shares the input part with the feature point detection branch. After two convolutions, the feature dimension is changed to W / 8×H / 8×256. A semi-dense descriptor can be obtained through training to reduce computational overhead. After that, bicubic polynomial interpolation is performed to obtain the descriptor information of the remaining positions. Then, a unified length description is achieved through L2-normalizes to obtain the descriptor corresponding to each pixel of the input image. Finally, only the feature points and their corresponding 256-dimensional descriptors are retained.

[0020] The loss function of the network is as follows:

[0021] L(X,X′,D,D′;Y,Y′,S)=LP (X, Y)+L P (X′, Y′)+λL d (D, D′, S) (1)

[0022] It consists of two parts, L p is the loss function of the feature point detection position, L d is the loss function of the descriptor part. Among them, X and D are the two output parts of the network, namely the feature point feature map and the descriptor feature map, Y is the position label of the feature point in the original image, X′, D′| and Y′ correspond to the results of the original image after projection transformation, and λ is an empirical value used to balance the loss value, usually 0.0001.

[0023] The loss function for the feature point detection part is:

[0024]

[0025] Among them, H c =H / 8,W c =W / 8, which means 1 / 8 of the original image size, x hw ,y hw Represent the values ​​of X and Y at (h, w) respectively.

[0026] The training process of the network is divided into three parts, including the generation and training of artificial data sets, self-labeling of real data feature points, and simultaneous training of feature points and descriptors.

[0027] ② In the face of the current situation that the overall number of multimodal MRI images is small and the annotation is difficult, the present invention first needs to artificially synthesize a large number of data sets. The characteristics of these data include: simple shape, small size and obvious feature points, such as points in plane geometry, end points and intersections of lines, vertices of triangles or quadrilaterals, vertices of cubes in solid geometry, etc. Different geometric figures are synthesized into one picture, and interference conditions such as blur and noise are appropriately added. In the process of training the artificial data set, a random homography matrix transformation is performed on the input synthetic data set, and the descriptor extraction branch does not participate in the training. Only the model's ability to extract feature points is trained. The model trained with a large number of artificial data sets has better detection performance in synthetic shapes, especially the detection ability of diagonal points, than traditional detectors, such as FAST, Harris, etc. However, the repeatability of such a model in real scenes is far inferior to that of traditional manual design methods. It is easy to fail to detect the originally detected hot spots during the change of viewing angle. Therefore, an adaptive transformation method is introduced in the annotation of the second part of real data.

[0028] The second part of the training process is to use the model trained in the artificial data set to automatically label the feature points of the data set. In order to improve the repeatability of the model, that is, the rotation invariance and scale invariance mentioned in the traditional method, the randomly generated homography matrix is ​​used to transform the image. The transformations involved include: displacement, scaling, rotation, projection, cropping, etc. This process is called adaptive transformation. The number of random homography matrices used is usually about 100, so that images under different perspectives are obtained. After that, feature points are extracted from the images under all perspectives to obtain a set of feature points under all perspectives. Then, the coordinates of the feature points in the set are inversely transformed. The coordinate set after inverse transformation is the coordinate label of the feature points of the real data set extracted by the network. Since some feature points are repeatedly detected under multiple perspectives, feature points will be clustered. Therefore, non-maximum suppression should be used for these clustered feature points to retain only the points with the highest probability in the area. The filtered area is usually a 4×4 pixel area. After that, the training and labeling work is repeated using the automatically labeled data set to enhance the performance of the model on real data. Generally, one or two repetitions can achieve good results.

[0029] Finally, using the real multimodal MRI image data set and feature point coordinate labels, the descriptor branch is introduced for joint training, and the loss value L of the original image feature point is calculated at the same time. p (X, Y), the transformed image feature point loss and the descriptor loss value L between the two d (X, Y). The trained network can simultaneously output feature point coordinates and descriptor information, and obtain a network model that is highly robust when illumination and viewing angle change. This method can be used to achieve real-time stitching of ordinary small-size images, but when applied to the stitching task of linear array hyperspectral data, the computing resources consumed by directly using this method are too large. The detection speed can be effectively improved by appropriately reducing the number of convolutional layers of the coding layer, but this will cause the performance of the network to deteriorate. Therefore, the coding part is reconstructed by re-parameterization in the present invention. The structural diagram of the model is shown in the figure. Figure 1 shown.

[0030] ③ The accurate calculation of the degree of extramural invasion of middle and low rectal cancer depends on clear and accurate target area delineation. The application of AI technology can greatly improve efficiency. The high accuracy of AI intelligent target area delineation can largely avoid ineffective treatment due to inaccurate target area delineation. At present, AI+target area delineation has been successfully used in lung cancer, breast cancer, nasopharyngeal cancer, liver cancer, prostate cancer, esophageal cancer and skin cancer. The specific implementation plan for intelligent delineation of middle and low rectal cancer in the present invention is as follows:

[0031] The method of the present invention relies on some real data support. In this embodiment, data collection and preprocessing need to collect 300 patients diagnosed with middle and low rectal cancer by pathological diagnosis from the hospital HIS and PACS systems, and manually outlined by more than 2 professional doctors to ensure the authenticity and reliability of the data. For data from different sources, data cleaning and labeling are required. Cleaning the data can eliminate noise, label its specific source, ensure accuracy and reliability, and facilitate subsequent verification and traceability; secondly, image feature extraction and selection are performed, and the image is preprocessed such as standardization, enhancement, and geometric correction to improve the stability of subsequent analysis; the ASPP module is selected to capture image context information at different scales, including multiple parallel void convolution branches and global average pooling. The feature maps of different branches are fused to form the final semantic segmentation feature map, and the loss function here is intended to use the cross entropy loss function. Finally, according to the model structure, an intuitive tumor outline image is generated, and quantitative indicators are provided to carry out the calculation of the degree of extramural invasion of middle and low rectal cancer to assist doctors in understanding and diagnosis, and the system is deployed in the actual hospital environment, and optimized according to feedback to ensure that the system runs stably and meets the needs of doctors.

Claims

1. An automatic delineation method for the extent of extramural invasion of mid- and low-level rectal cancer based on multimodal MRI. It is characterized in that The following steps are involved: (1) An unsupervised network was developed to simultaneously detect feature points and generate descriptors on full-size images, which was used to match and fusion T2WI and DWI images in MRI of low and middle rectal cancer. (2) A miniaturized method for constructing a database of the degree of extramural invasion of low and middle rectal cancer was developed. (3) A target area delineation network for the degree of extramural invasion of low and middle rectal cancer was designed.

2. The method according to claim 1, It is characterized in that The model structure of the network is divided into the encoding part and the decoding part. In the encoding part, the input is the grayscale image of batch 1, which consists of a convolution layer, a Max-Pooling layer, and a nonlinear activation layer. After 8 convolution operations and three Max-Pooling layers, the image size is reduced to 1 / 8 of the original size, and the image changes from W×H×1 to W / 8×H / 8×256. This paper mainly reparameterizes this part; in the decoding part, there are two branches: feature point extraction branch and descriptor generation branch. The feature point extraction branch outputs the probability information of each pixel in the original image as a feature point, with a value between 0 and 1, while the descriptor generation branch outputs the descriptor corresponding to each pixel, and each descriptor consists of a 256-dimensional floating-point feature vector. The feature point detection branch of the decoding part first performs two convolution operations to obtain the parameters of 64 channels, and then adds a non-feature point channel to prevent the abnormal randomness of the response on each channel after softmax due to noise interference. Then, the pixels are shuffled along the 64 channels. An 8×8 cell block can be obtained for every 64 pixels. The cell blocks are spliced ​​together to form a feature point probability matrix with the same size as the input image. The value of each pixel corresponds to the probability value of the pixel coordinate of the input image as a feature point. The larger the probability value, the higher the possibility that the corresponding position of the input image is a feature point. After that, the non-maximum suppression and edge suppression methods can be used to remove the dense points of repeated detection and the erroneous detection points at the edge position, and then the threshold is used to filter out the points with higher probability to obtain the feature point coordinates with uniform distribution and pixel-level positioning. The descriptor generation branch shares the input part with the feature point detection branch. After two convolutions, the feature dimension is changed to W / 8×H / 8×256. A semi-dense descriptor can be obtained through training to reduce computational overhead. After that, bicubic polynomial interpolation is performed to obtain the descriptor information of the remaining positions. Then, a unified length description is achieved through L2-normalizes to obtain the descriptor corresponding to each pixel of the input image. Finally, only the feature points and their corresponding 256-dimensional descriptors are retained.

3. The method according to claim 1, It is characterized in that The database consists of two parts: an artificially synthesized dataset and a doctor-annotated dataset.

4. The method according to claim 3, It is characterized in that The characteristics of artificially synthesized data sets include: points in plane geometry, endpoints and intersections of lines, vertices of triangles or quadrilaterals, vertices of cubes in solid geometry, etc. Different geometric figures are synthesized into one picture, and interference conditions such as blur and noise are appropriately added. In the process of training the artificial data set, a random homography matrix transformation is performed on the input synthetic data set. The descriptor extraction branch does not participate in the training, and only the model's ability to extract feature points is trained.

5. The method according to claim 3, It is characterized in that The features of the doctor's annotation dataset include: adaptive transformations such as displacement, scaling, rotation, projection, and cropping of the basic image. The number of random homography matrices used is usually 100, so as to obtain images under different perspectives, and extract feature points from images under all perspectives to obtain a set of feature points under all perspectives, and then perform an inverse transformation on the feature point coordinates in the set. The inverse transformed coordinate set is the real dataset feature point coordinate label extracted by the network. Since some feature points are repeatedly detected under multiple perspectives, feature points will cluster. Therefore, non-maximum suppression should be used for these clustered feature points to retain only the points with the highest probability in the area. The filtered area is usually a 4×4 pixel area. Repeat the training and annotation work with the automatically annotated dataset to enhance the performance of the model on real data. Generally, one or two repetitions can achieve good results.

6. The method according to claim 1, It is characterized in that Data collection and preprocessing requires collecting more than 300 patients diagnosed with middle and low rectal cancer by pathological diagnosis from the hospital's HIS and PACS systems, and manually outlining by more than 2 professional doctors to ensure the authenticity and reliability of the data. Data from different sources need to be cleaned and labeled. Cleaning the data can eliminate noise, label its specific source, and ensure accuracy and reliability; the ASPP module is used to capture image context information at different scales, including multiple parallel dilated convolution branches and global average pooling; the feature maps of different branches are fused to form the final semantic segmentation feature map. The loss function here is intended to use the cross entropy loss function; based on the model structure, an intuitive tumor outline image is generated.

7. An automatic delineation method for the extent of extramural invasion of mid- and low-level rectal cancer based on multimodal MRI. It is characterized in that The method for automatically outlining the extent of extramural invasion of middle and low rectal cancer based on multimodal MRI is used to execute the method for automatically outlining the extent of extramural invasion of middle and low rectal cancer based on multimodal MRI as described in any one of claims 1-6.

Citation Information

Cited By

  • Automatic sketching method for rectal cancer radiotherapy target area and related equipment

    CN119313678A