Enhanced CT image rectal cancer staging auxiliary diagnosis system based on deep learning

Abstract

The present invention relates to an enhanced CT image rectal cancer staging auxiliary diagnosis system based on deep learning, including an enhanced CT image input module, an enhanced CT image and annotation database, an image preprocessing module, a rectal lesion area discrimination module, a lesion lymph node recognition module, a lesion feature extraction module, a comprehensive diagnosis module, and a visualization module. It is an auxiliary diagnosis device. Aiming at the problem that the rectal enhanced CT image data structure is complex and it is difficult to distinguish the cancerous area and its staging, the enhanced CT image rectal cancer data annotation and data set construction, rectal cancer lesion area discrimination based on the self-attention deep learning model, and metastatic lymph node recognition based on sequence adaptive feature fusion are studied. The intelligent auxiliary diagnosis system for rectal cancer staging is designed and implemented, and clinical application experiments are conducted to verify it, so as to improve the accuracy and efficiency of comprehensive data collection before rectal cancer surgery.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image processing, and in particular relates to an enhanced CT image rectal cancer staging auxiliary diagnosis system based on deep learning. Background Art

[0002] Rectal cancer is the most common malignant tumor of the digestive system. It generally refers to the cancer between the dentate line and the junction of the rectum and sigmoid colon. It is one of the most common malignant tumors of the digestive tract. Rectal cancer is located low and can be easily diagnosed by rectal examination and colonoscopy. However, due to its deep location in the pelvic cavity, complex anatomical relationships, and limitations in examination methods, it is difficult to achieve accurate preoperative staging diagnosis. The staging diagnosis of rectal cancer is crucial for accurate individualized treatment plan decisions and is the prerequisite for ensuring a good prognosis for patients. X-ray, MRI and CT are currently the most commonly used imaging technologies. X-ray technology is more suitable for specific bone organs, but it is not clear for tissue and organ examinations. Barium enema is generally used to diagnose rectal cancer, but it can only reflect the location of the lesion and the information in the intestinal lumen of the lesion segment. MRI technology has a high soft tissue resolution, but the high price and accessibility of equipment limit its widespread clinical application to a certain extent, and the examination time and imaging time are long. Among various rectal cancer screening methods, CT examination is currently the most popular and effective method. CT is also the most widely used colorectal cancer imaging examination method in clinical practice. In rectal cancer detection, enhanced CT images can clearly show the invasion of the external muscle layer and surrounding organs by the tumor, and the size and morphological characteristics of the lymph nodes can also be presented. Therefore, CT images have become an important reference for the staging diagnosis of rectal cancer. By constructing a rectal cancer staging diagnosis data set for massive medical imaging data, and building and training deep learning neural networks, benign, malignant and staged rectal tumors can be identified, which can partially replace the work of radiologists, realize the rapid and accurate identification of the clinical TN stage of rectal cancer, and improve work efficiency.

[0003] The Chinese patent "Deep Learning Colorectal Cancer Polyp Segmentation Device Based on Enhanced Multi-Scale Features" (Application No.: 202110879728.6, Application Date: 2021.08.02) discloses a deep learning colorectal cancer polyp segmentation device based on enhanced multi-scale features. The scheme first adjusts and normalizes the resolution of colorectal cancer polyp image data; uses a feature extractor with a multi-scale residual structure and a receptive field block component that can capture a multi-scale receptive field to extract the diversity features of the polyp; then uses dense multi-scale jump connections to transfer context information to achieve segmentation details, and completes boundary segmentation through the attention mechanism provided by the local context; and uses deep supervision technology to calibrate during the upsampling process to reduce the gradient vanishing or explosion phenomenon during training. It solves the problem that small polyps are difficult to distinguish and locate, and the boundary between polyps and surrounding tissues is unclear. At the same time, the introduction of the deep supervision mechanism optimizes the gradient of the network model, accelerates the convergence of the network model, and shortens the training time of the network model. The experimental results verify the feasibility of using the attention mechanism provided by the local context for computer-aided diagnosis. Selecting appropriate model parameters and model structure can achieve good recognition results. However, since there are many adjacent organs and blood vessels around the rectum, and the angle and shape of the rectum vary greatly from person to person, the rectal CT structure is more complex. Directly using these mature auxiliary diagnosis devices and methods lacks the use of the overall global information of enhanced CT images, cannot fully utilize the global dependency between features for modeling, and has a low accuracy rate in detecting lesions. It is also easy to ignore the information of weak targets such as lymph nodes, reducing the reliability of staging diagnosis results. Summary of the invention

[0004] In view of the defects of the above-mentioned prior art, the purpose of the present invention is to provide an enhanced CT image rectal cancer staging auxiliary diagnosis system based on deep learning, which is also an auxiliary diagnosis device. In view of the problem that the rectal enhanced CT image data structure is complex and it is difficult to distinguish the cancerous area and its staging, the enhanced CT image rectal cancer data annotation and data set construction, the recognition of rectal cancer lesion area based on self-attention deep learning model, and the identification of metastatic lymph nodes based on sequence adaptive feature fusion are studied. An intelligent auxiliary diagnosis system for rectal cancer staging is designed and implemented, and clinical application experiments are carried out to verify it, so as to improve the accuracy and efficiency of comprehensive data collection before rectal cancer surgery.

[0005] In order to achieve the above objectives, the technical solution adopted by the present invention is: an enhanced CT image rectal cancer staging auxiliary diagnosis system based on deep learning, including an enhanced CT image input module, an enhanced CT image and annotation database, an image preprocessing module, a rectal lesion area discrimination module, a lesion lymph node recognition module, a lesion feature extraction module, a comprehensive diagnosis module, and a visualization module;

[0006] The enhanced CT image input module is used to input the rectal enhanced CT image;

[0007] The enhanced CT image and annotation database is used to store the rectal CT images input by the CT image input module, the enhanced CT rectal cancer image data set and the annotation data set;

[0008] The image preprocessing module is used to perform noise reduction and image enhancement processing on the enhanced CT image;

[0009] The rectal lesion region identification module is used to identify a suspected rectal tumor region and segment the region;

[0010] The diseased lymph node identification module is used to identify periintestinal diseased lymph nodes;

[0011] The lesion feature extraction module is used to identify and collect rectal tumor lesion area feature data, and count the number of periintestinal lesion lymph nodes to form a rectal lesion feature parameter set;

[0012] The comprehensive diagnosis module is used to integrate rectal lesion area discrimination information, lesion lymph node identification information, and inspection object site characteristics, and compare with the TN staging prior knowledge base to provide a rectal cancer TN staging auxiliary diagnosis result;

[0013] The visualization module is used to display the input enhanced CT image and mark the information related to the characteristics of the rectal tumor lesion area on the image.

[0014] Furthermore, the lesion feature extraction module identifies and collects characteristic parameters of the rectal tumor lesion area including: the wall thickness of the rectal tumor lesion area, the density difference between the tumor and the normal wall, whether there are burr-like protrusions, and whether adjacent structures are involved.

[0015] Furthermore, the visualization module is used to display information related to the regional characteristics of rectal tumor lesions marked on the image, including the concentrated location of rectal cancer, the thickness of the wall at the lesion, the burrs around the lesion, the involvement of adjacent structures, and the diseased lymph node information and TN staging results.

[0016] Furthermore, the system operation execution includes the following steps:

[0017] S1. Construct enhanced CT rectal cancer image dataset and annotation database;

[0018] S2. Perform format conversion and image noise reduction on enhanced CT images;

[0019] S3. The rectal lesion area discrimination module uses the training data in the enhanced CT rectal cancer image dataset and the annotation database to train a self-attention-based deep learning model to determine whether the input enhanced CT image contains a suspected rectal tumor shape area and segment the lesion area;

[0020] S4. The diseased lymph node recognition module uses a sequence adaptive feature fusion method to identify periintestinal diseased lymph nodes based on CT image lymph node features;

[0021] S5. The lesion feature extraction module extracts the wall thickness of the rectal tumor lesion area, the density difference between the tumor and the normal wall, the burr-like protrusions around the lesion, and the appearance characteristics of the affected adjacent structures, and counts the number of peri-intestinal lesion lymph nodes to form a set of rectal lesion feature parameters;

[0022] S6. The comprehensive diagnosis module integrates the rectal tumor lesion area discrimination information, diseased lymph node identification information, and lesion feature information, and combines it with the TN staging prior knowledge base to achieve clinical TN staging auxiliary diagnosis of rectal cancer.

[0023] Furthermore, in step S1, the step of constructing the enhanced CT rectal cancer image data set and annotation database includes:

[0024] S11. Specify a data annotation format to assign different labels to rectal cancer lesions and perianal suspicious and normal lymph nodes corresponding to different rectal cancer stages;

[0025] S12. Use image annotation tools, combine the pathology knowledge base and enhanced CT image features, and annotate the enhanced CT images in accordance with the prescribed data annotation format;

[0026] S13. Store the annotation file and the original image file in correspondence with each other, and store the corresponding necessary object information to construct a data set.

[0027] Furthermore, in step S2, image data preprocessing is performed, the steps comprising:

[0028] S21. The enhanced CT rectal cancer image data stored in the original medical image in the form of dicom image series is uniformly converted into NiFTI data format, so that the original CT image data format is consistent with the corresponding annotated label data format;

[0029] S22. Use the medical image processing library based on the deep learning framework to read the converted NiFTI data, convert the CT rectal cancer image data and the annotated label data into a tensor data structure processed by the deep learning framework, and establish a mapping relationship between the original image data and the label data;

[0030] S23. Perform image resampling, noise reduction, random affine transformation, and image channel number addition operations on the image data and label data that establish and map the relationship, so as to achieve image enhancement.

[0031] Furthermore, in step S3, the rectal lesion area discrimination module adopts a self-attention deep learning model to construct a global dependency relationship of CT image features using a self-attention mechanism to capture the overall internal correlation of image data and image features; wherein the suspected lesion area discrimination steps based on the self-attention mechanism are:

[0032] S31. First, the preprocessed CT image is input into the feature extraction network, and a feature map of the corresponding depth level is obtained through one or more downsampling and pooling operations;

[0033] S32. Introduce the attention mechanism, compress each channel of the feature map accordingly, pass it through the fully connected layer, and use the activation function to obtain the importance of different channels, and convert it into an attention vector;

[0034] S33. Integrate the obtained feature importance weights into the original deep learning network structure feature map, further guide the focus of the network, and realize the integration of the attention mechanism;

[0035] The specific formula is as follows:

[0036] A=Att(X,θ)=δ(W2δ(W1GAP(X)))# (1)

[0037] Y=AX# (2).

[0038] Furthermore, in step S4, the identification of the diseased lymph node identification module adopts a method based on sequence adaptive feature fusion, and uses continuous multi-frame image annotation information to identify small lymph node targets; wherein the lymph node diagnosis processing steps based on sequence adaptive feature fusion include:

[0039] S41. Comprehensively classify and judge the fused sequence features and feature information of multi-frame data to obtain the location and information of suspicious lymph nodes in the feature map;

[0040] S42. Map the lymph node location information in the feature map back to the original enhanced CT image.

[0041] Furthermore, in step S6, the comprehensive diagnosis module integrates information on rectal lesions, lymph node metastasis, and lesion characteristics, and makes a judgment by comparing the data in the medical pathology knowledge base:

[0042] S61. normalizing the suspicious lesion location information, suspicious lymph nodes, and appearance feature information extracted by the feature extraction network to achieve consistency in data format;

[0043] S62. Through the corresponding fusion network, the semantic differences between different information are eliminated to achieve the splicing and fusion of features;

[0044] S63. According to the comparison with the corresponding clinical knowledge base, the integrated information is classified and judged accordingly to achieve a comprehensive judgment of the information on rectal cancer;

[0045] S64. Map the judgment result of S63 back to the original enhanced CT image to achieve corresponding TN labeling and visualization.

[0046] Furthermore, in step S11, four different labels are specified for the intestinal wall, and the correspondence between the stages and the labels is as follows: T0-label1, T1-label2, T2-label3, T3-label4, T4-label5; two different labels are specified for the lymph nodes, and the correspondence between the status and the label is: normal lymph nodes-label6, suspicious lymph nodes-label7.

[0047] The technical effects of the present invention are:

[0048] (1) The present invention constructs a new and effective enhanced CT image dataset of rectal cancer from scratch, collects preoperative enhanced CT image data of rectal cancer at different stages, and its pathological (etc.) data information, annotates the image data with cancerous areas, lymph node metastasis areas, and staging diagnosis information, and establishes a rectal cancer dataset.

[0049] (2) The present invention studies an image segmentation network based on a self-attention deep learning model to achieve the discrimination of rectal lesion areas. The self-attention mechanism that uses the weighted sum of features at all positions to calculate the response shows good performance in modeling global dependencies, which is conducive to capturing the overall internal correlation of image data and image features, thereby improving the segmentation accuracy of the lesion area and making it more conducive to discriminating the lesion area.

[0050] (3) The present invention proposes a deep learning model for identifying small and weak lymph node targets using continuous multi-frame image annotation information. By learning the continuous multi-frame lymph node annotation information of the same patient, the multiple lymph node annotation information is fused using an adaptive method, thereby realizing the identification of small and weak targets of metastatic lymph nodes.

[0051] (4) The present invention has the support of hospitals in image acquisition and can obtain a large number of high-quality thin-layer enhanced CT images of rectal cancer. Based on the knowledge base of pathology, the present invention takes into account both rectal images and surrounding lymphatic images during processing, and uses the information of both to complete the judgment of rectal cancer, thereby improving the accuracy of diagnosis.

[0052] (5) The present invention introduces human-computer interaction into the computer-aided diagnosis device. When using it, the doctor can interact with the device system of the present invention. In the segmentation and detection stage, the doctor can modify the output results of the link more accurately. In the classification stage, rewards are given to the correctly classified results, and penalties are given to the incorrectly classified results. The system then conducts reinforcement learning based on the feedback results and gradually achieves perfect performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is a structural block diagram of a device for assisting in the diagnosis of rectal cancer staging based on CT images based on deep learning according to the present invention;

[0054] Figure 2 A file structure diagram of the data set constructed in step S12 for executing the work of the present invention;

[0055] Figure 3 A specific implementation diagram of the deep learning model based on self-attention in step S33 of the present invention;

[0056] Figure 4 A network block diagram of a self-attention based deep learning model in step S34 for executing the work of the present invention;

[0057] Figure 5 The suspicious lymphatic network structure diagram based on sequence fusion in step S43 is executed for the work of the present invention;

[0058] Figure 6 The schematic diagram of visual output of the result in step S64 of the present invention is shown. DETAILED DESCRIPTION

[0059] Reference Figure 1 The present invention provides an enhanced CT image rectal cancer staging auxiliary diagnosis system based on deep learning, including an enhanced CT image input module, an enhanced CT image and annotation database, an image preprocessing module, a rectal lesion area discrimination module, a lesion lymph node recognition module, a lesion feature extraction module, a comprehensive diagnosis module, and a visualization module.

[0060] The execution of the device includes the following steps:

[0061] S1. Construct enhanced CT rectal cancer image dataset and annotation database;

[0062] S2. Perform format conversion and image noise reduction on enhanced CT images;

[0063] S3. The rectal lesion region discrimination module is based on the self-attention deep learning model to determine whether the CT image contains a suspected rectal tumor (lesion) shape area and segment the lesion area;

[0064] S4. The diseased lymph node recognition module uses a sequence adaptive feature fusion method to identify periintestinal diseased lymph nodes based on CT image lymph node features;

[0065] S5. The lesion feature extraction module identifies and collects the (lesion) appearance feature data of the tube wall thickness in the rectal tumor lesion area, the density difference between the tumor and the normal tube wall, whether there are burr-like protrusions and the involvement of adjacent structures (etc.), and counts the number of periintestinal lesion lymph nodes to form a rectal lesion feature parameter set;

[0066] S6. The comprehensive diagnosis module integrates the rectal tumor lesion area discrimination information, lesion lymph node identification information, and lesion feature information, and compares it with the TN staging prior knowledge base to achieve TN staging auxiliary diagnosis of rectal cancer;

[0067] The visualization module is used to display the input enhanced CT images and mark the rectal cancer wall thickness, spicule-like protrusions, involvement of adjacent structures, diseased lymph nodes (etc.) information and TN staging results on the images.

[0068] Compared with the existing technology, the device of the present invention can quickly and accurately identify the rectal tumor lesion area and the diseased lymph node metastasis, quickly determine the clinical TN stage of rectal cancer, and provide a reference for intelligent screening of rectal cancer CT images. It can effectively improve the accuracy and efficiency of comprehensive data collection before rectal cancer surgery, help doctors find problems in time, and determine the corresponding TN stage. It provides a more reliable scientific basis for the standardized operation of clinical physicians.

[0069] Furthermore, a deep learning-based enhanced CT image rectal cancer staging auxiliary diagnosis system includes an enhanced CT image input module, an enhanced CT image and annotation database, an image preprocessing module, a rectal lesion area discrimination module, a lesion lymph node recognition module, a lesion feature extraction module, a comprehensive diagnosis module, and a visualization module. Among them:

[0070] The enhanced CT image input module is used to input the rectal enhanced CT image;

[0071] The enhanced CT image and annotation database is used to store the rectal CT images input by the CT image input module, the enhanced CT rectal cancer image data set and the annotation data set;

[0072] The image preprocessing module is used to perform noise reduction and image enhancement processing on the enhanced CT image;

[0073] The rectal lesion region identification module is used to identify a suspected rectal tumor region and segment the region;

[0074] The diseased lymph node identification module is used to identify periintestinal diseased lymph nodes;

[0075] The lesion feature extraction module is used to identify and collect feature data of the wall thickness of the rectal tumor lesion area, the density difference between the tumor and the normal wall, whether there are burr-like protrusions and the involvement of adjacent structures (etc.), and count the number of periintestinal lesion lymph nodes to form a rectal lesion feature parameter set;

[0076] The comprehensive diagnosis module is used to integrate rectal lesion area discrimination information, lesion lymph node identification information, and inspection object site characteristics, and compare with the TN staging prior knowledge base to provide a rectal cancer TN staging auxiliary diagnosis result.

[0077] The visualization module is used to display the input enhanced CT images and mark the concentrated location of rectal cancer (lesions), the thickness of the tube wall at the lesion, the burr protrusions around the lesion, the involvement of adjacent structures and the diseased lymph nodes (etc.) and the TN staging results on the images;

[0078] The enhanced CT image rectal cancer staging auxiliary diagnosis system based on deep learning of the present invention comprises the following steps:

[0079] S1. Construct enhanced CT rectal cancer image dataset and annotation database;

[0080] S2. Perform format conversion and image noise reduction on enhanced CT images;

[0081] S3. The rectal lesion area discrimination module uses the training data in the enhanced CT rectal cancer image dataset and the annotation database to train a self-attention-based deep learning model to determine whether the input enhanced CT image contains a suspected rectal tumor (lesion) shape area and segment the lesion area;

[0082] S4. The diseased lymph node recognition module uses a sequence adaptive feature fusion method to identify periintestinal diseased lymph nodes based on CT image lymph node features;

[0083] S5. The lesion feature extraction module identifies and collects the (lesion) appearance feature data of the tube wall thickness in the rectal tumor lesion area, the density difference between the tumor and the normal tube wall, the burr-like protrusions around the lesion, and the involvement of the adjacent structure (etc.), and counts the number of lymph nodes around the intestine to form a rectal lesion feature parameter set;

[0084] S6. (Comprehensive diagnosis module) integrates the rectal tumor lesion area discrimination information, diseased lymph node identification information, and lesion feature information, and combines it with the TN staging prior knowledge base (comparison) to achieve clinical TN staging auxiliary diagnosis of rectal cancer.

[0085] Wherein, in step S1, the steps of constructing the enhanced CT rectal cancer image data set and annotation database include:

[0086] S11. Specify a data annotation format to assign different labels to rectal cancer lesions and perianal suspicious and normal lymph nodes corresponding to different rectal cancer stages;

[0087] S12. Use image annotation tools, combine the pathology knowledge base and enhanced CT image features, and annotate the enhanced CT images in accordance with the prescribed data annotation format;

[0088] S13. Store the annotation file and the original image file in correspondence with each other, and store the corresponding necessary object information to construct a data set.

[0089] In step S2, image data preprocessing is performed, and the steps include:

[0090] S21. The enhanced CT rectal cancer image data stored in the original medical image in the form of dicom image series is uniformly converted into NiFTI data format, so that the original CT image data format is consistent with the corresponding annotated label data format;

[0091] S22. Use the medical image processing library based on the deep learning framework to read the converted NiFTI data, convert the CT rectal cancer image data and the annotated label data into a tensor data structure processed by the deep learning framework, and establish a mapping relationship between the original image data and the label data;

[0092] S23. Perform image resampling, denoising, random affine transformation, and adding channels to the image size (etc.) on the image data and label data that establish and map the relationship, so as to achieve image enhancement and improve the performance of subsequent deep learning network models.

[0093] Furthermore, in step S3, the rectal lesion area discrimination module adopts a self-attention deep learning model to construct the global dependency of CT image features using the self-attention mechanism, and captures the overall internal correlation of image data and image features, thereby improving the segmentation accuracy of the lesion area and the lesion area discrimination accuracy. The steps of suspected lesion area discrimination based on the self-attention mechanism are as follows:

[0094] S31. First, the preprocessed CT image is input into the feature extraction network, and a feature map of the corresponding depth level is obtained through one or more downsampling and pooling operations;

[0095] S32. Introduce a self-attention mechanism to obtain global feature information of CT images, capture channel relationships, and improve feature representation capabilities; compress each channel of the feature map accordingly, and after compression, obtain the importance of different channels through a fully connected layer, activation function (etc.) operations, and convert them into attention vectors, which can better help the model distinguish the rectal wall area from the background organs, and facilitate the measurement of rectal wall thickness;

[0096] S33. The obtained feature importance weights are integrated into the original deep learning network structure feature map to further guide the network to focus on the lesion area and realize the integration of the attention mechanism. The specific formula is as follows:

[0097] A=Att(X,θ)=δ(W2δ(W1GAP(X)))# (1)

[0098] Y=AX# (2)

[0099] Among them, A is the corresponding dimension weight obtained after attention calculation; Att() is the attention calculation operation function, X is the feature map data extracted by the feature extraction convolutional network, and θ is the network parameter. δ is the ReLu activation function, which is used to provide nonlinear gating operations in the network. W1 and W2 correspond to two fully connected layers to achieve feature dimensionality reduction. GAP() is the global average pooling operation function; Y is the feature (function value) of feature map X after calculation by the attention module.

[0100] That is, constantly adjust the network's attention, and in repeated tests, guide the network to further focus on the lesion area, so as to improve the network's ability to distinguish suspected lesion areas.

[0101] Furthermore, in step S4, the identification of the diseased lymph node recognition module adopts a method based on sequence adaptive feature fusion, using continuous multi-frame image annotation information to identify weak and small lymph node targets, solving the problem of weak and small lymph node targets. Among them, the lymph node diagnosis processing steps based on sequence adaptive feature fusion include:

[0102] (Multiple sequence feature maps are fused into one feature map through the feature extraction network, and voxel information, continuous change information between sequences, and similarity information are extracted accordingly)

[0103] S41. Comprehensively classify and judge the fused sequence features and feature information of multi-frame data to obtain the location and information of suspicious lymph nodes in the feature map;

[0104] S42. Map the lymph node location (etc.) information in the feature map back to the original enhanced CT image, thereby locating the suspicious lymph node location in the original image.

[0105] Furthermore, in step S6, the comprehensive diagnosis module integrates information on rectal lesions, lymph node metastasis, lesion characteristics (etc.), and makes a judgment by comparing the data of the medical pathology knowledge base:

[0106] S61. normalizing the suspicious lesion location information, suspicious lymph nodes, and (lesion) appearance feature information extracted by the feature extraction network to achieve consistency in data format;

[0107] S62. Eliminate semantic differences between different information through corresponding fusion networks, and realize splicing and fusion of features;

[0108] S63. According to the comparison with the corresponding clinical knowledge base, the integrated information is classified and judged accordingly to achieve a comprehensive judgment of the information on rectal cancer;

[0109] S64. Map the judgment result of S63 back to the original enhanced CT image to achieve corresponding TN labeling and visualization.

[0110] The technical solutions of various embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0111] The present invention provides a deep learning-based enhanced CT image rectal cancer staging auxiliary diagnosis system, which is a device for taking rectal CT images as analysis objects. It performs auxiliary diagnosis by constructing and training an image segmentation network based on a self-attention deep learning model and a deep learning model for identifying small lymph node targets using continuous multi-frame image annotation information. It can quickly and accurately give the range of the lesion and obtain the rectal cancer staging results.

[0112] The technical principle is based on the fact that normal rectal CT images and CT images containing lesion areas differ in image grayscale, intestinal wall thickness and shape, size and morphology of periintestinal lymph nodes. Different stages of cancer also have certain differences in grayscale, shape and other characteristics. Based on this difference, image feature extraction and the trained deep learning neural network are used to effectively identify CT images. For example, T1 stage tumors invade the submucosal layer but do not involve the muscularis propria, which is manifested as localized enhanced lesions within the submucosal layer. The lesions are soft tissue density masses with clear boundaries, no intestinal wall thickening, and the outer edge of the intestinal wall is smooth and clear. T2 stage tumors invade the muscularis propria, and the tumors are confined to the intestinal wall. The important feature is that the external muscular layer is not invaded, which is manifested as localized enhanced lesions in the rectal wall. The tumors vary in size and are often lobed and asymmetric. When the long axis of the tumor is consistent with the scanning plane, the intestinal wall is tubular and irregularly thickened, stiff, and narrow, and the outer edge of the intestinal wall is smooth and clear. In the T3 stage, the tumor grows through the outer muscle layer to the surrounding mesorectum, which is manifested as a rough outer edge of the intestinal wall muscle layer. The spread to the mesorectal fat can be manifested as a needle-like increase in density in the mesorectal fat, or deformation and distortion of the muscular layer. Low-density ischemic necrosis areas may appear in larger tumors. In the T4 stage, the tumor invades surrounding structures such as peritoneal reflections, pelvic walls, vagina, prostate, bladder or seminal vesicles, which is manifested as increased density, speckled, cord-like shadows and soft tissue shadows in the surrounding fat tissue, or no gaps in the fat between adjacent organs. Low-density ischemic necrosis areas may appear in larger tumors. The N stage uses lymph node short diameter > 5mm, blurred boundaries, uneven density, enhanced performance, and clustered lymph nodes ≥ 3 as positive criteria. These are mainly reflected in the image features as differences in texture features and shape features. Texture features are formed by the distribution of grayscale features according to a certain rule. Regions with the same texture features can be expressed as shape features. Based on the differences in texture features, especially the texture features at the shape boundaries, it can be determined whether there is a suspected lesion area. The type of lesion can be further identified based on the differences in texture features and shape features.

[0113] Based on the above identification basis, the present invention proposes a deep learning-based enhanced CT image rectal cancer staging auxiliary diagnosis system. Figure 1 As shown. It includes: enhanced CT image input module, enhanced CT image and annotation database, image preprocessing module, rectal lesion area discrimination module, lesion lymph node recognition module, lesion feature extraction module, comprehensive diagnosis module, and visualization module, among which:

[0114] The enhanced CT image input module is used to input the rectal enhanced CT image;

[0115] The enhanced CT image and annotation database is used to store the rectal CT images input by the CT image input module, the enhanced CT rectal cancer image data set and the annotation data set;

[0116] The image preprocessing module is used to perform noise reduction and image enhancement processing on the enhanced CT image;

[0117] The rectal lesion area identification module is used to identify suspected rectal tumor lesion areas and segment the lesion areas;

[0118] The diseased lymph node identification module is used to identify periintestinal diseased lymph nodes;

[0119] The lesion feature extraction module is used to identify and collect (lesion) appearance feature data such as the thickness of the tube wall in the rectal tumor lesion area, the density difference between the tumor and the normal tube wall, whether there are burr-like protrusions around the lesion, and whether there is involvement of adjacent structures, and count the number of periintestinal lesion lymph nodes to form a rectal lesion feature parameter set;

[0120] The comprehensive diagnosis module is used to integrate rectal lesion area discrimination information, lesion lymph node identification information, lesion characteristics, and compare with the TN staging prior knowledge base to provide a rectal cancer TN staging auxiliary diagnosis result.

[0121] The visualization module is used to display the input enhanced CT images and mark the rectal lesion wall thickness, perifocal burr-like protrusions, adjacent structure involvement, diseased lymph node lesion information and TN staging results on the images;

[0122] Furthermore, the rectal cancer staging auxiliary diagnosis system comprises the following steps:

[0123] S1. Construct enhanced CT rectal cancer image dataset and annotation database;

[0124] S2. Perform format conversion and image noise reduction on enhanced CT images;

[0125] S3. The rectal lesion area discrimination module uses the training data in the enhanced CT rectal cancer image dataset and the annotation database to train a self-attention-based deep learning model to determine whether the input enhanced CT image contains a suspected rectal tumor (lesion) shape area and segment the lesion area;

[0126] S4. The diseased lymph node recognition module uses a sequence adaptive feature fusion method to identify periintestinal diseased lymph nodes based on CT image lymph node features;

[0127] S5. The lesion feature extraction module identifies and collects the shape feature data of the rectal tumor lesion area, such as the wall thickness and burr protrusions, and counts the number of periintestinal lesion lymph nodes to form a rectal lesion feature parameter set;

[0128] S6. (Comprehensive diagnosis module) integrates the rectal tumor lesion area discrimination information, diseased lymph node identification information, and lesion feature information, and combines it with the TN staging prior knowledge base (comparison) to achieve TN staging auxiliary diagnosis of rectal cancer.

[0129] In step S1, the steps of constructing the enhanced CT rectal cancer image dataset and annotation database include:

[0130] S11. Specify the data annotation format, assign different labels to the rectal segments and perianal lymph nodes corresponding to different rectal cancer stages; specify 4 different labels for the intestinal wall, and the correspondence between the stages and labels is as follows: T0-label1, T1-label2, T2-label3, T3-label4, T4-label5. Specify two different labels for the lymph nodes, and the correspondence between the status and label is as follows: normal lymph nodes-label6, suspicious lymph nodes-label7;

[0131] S12. Use image annotation tools, combine the pathology knowledge base and enhanced CT image features, and annotate the enhanced CT images in accordance with the prescribed data annotation format;

[0132] S13. Store the annotation file and the original image file in correspondence, and store the corresponding patient information to construct a data set. The data set file structure is as follows: Figure 2 shown.

[0133] Furthermore, in step S2, the image data preprocessing step is:

[0134] S21. uniformly convert the enhanced CT image data stored in the form of dicom image series in the original medical image into NiFTI data format, so that the original CT rectal cancer image data format is consistent with the corresponding annotated label data format;

[0135] S22. Use the medical image processing library based on the deep learning framework to read the converted NiFTI data, convert the CT rectal cancer image data and the annotated label data into a tensor data structure processed by the deep learning framework, and establish a mapping relationship between the original image data and the label data;

[0136] S23. Perform image resampling, noise reduction, random affine transformation, increase the number of channels in the image size and other operations on the image data and label data that establish and map the relationship, so as to achieve image enhancement and improve the performance of the deep learning network.

[0137] Furthermore, in step S3, the suspected lesion area identification step based on the self-attention machine deep learning model is:

[0138] S31. First, the preprocessed data is input into the feature extraction convolutional network, and after a series of downsampling and pooling operations, the feature extraction of the enhanced CT image data is completed, and a feature map of the corresponding depth level is obtained;

[0139] S32. Introduce the self-attention mechanism to obtain the global feature information of CT images, capture channel relationships, and improve the feature representation capability. Compress each channel of the feature map accordingly, and obtain the importance of different channels through full connection layers, activation functions and other operations after compression, and convert them into attention vectors, which can better help the model distinguish the rectal wall area from the background organs, and facilitate the measurement of rectal wall thickness;

[0140] S33. The obtained feature importance weights are integrated into the original deep learning network structure feature map to further guide the network to focus on the lesion area and realize the fusion of the attention mechanism. The formula is as follows:

[0141] A=Att(X,θ)=δ(W2δ(W1GAP(X)))# (1)

[0142] Y=AX# (2)

[0143] Among them, A is the corresponding dimension weight obtained after attention calculation; Att() is the attention calculation operation function, X is the feature map data extracted by the feature extraction convolutional network, and θ is the network parameter. δ is the ReLu activation function, which is used to provide nonlinear gating operations in the network. W1 and W2 correspond to two fully connected layers to achieve feature dimensionality reduction. GAP() is the global average pooling operation function; Y is the feature (function value) of feature map X after calculation by the attention module.

[0144] The specific implementation form of the self-attention model is as follows Figure 3 As shown in [the figure, the feature map size obtained by the feature extraction network is N×H×W, where N is the channel dimension of the feature map, H is the height of the feature map, and W is the width of the feature map. B is the N-dimensional channel dimension weight vector obtained after the feature map is globally pooled. C is the feature map after full connection and ReLu activation function, and D is the feature map after C is calculated by attention and resized to N×H×W];

[0145] That is, the network's attention is constantly adjusted, and in repeated training, the network is guided to further focus on the lesion area, so as to improve the network's ability to distinguish the rectal tumor lesion area. The overall deep learning network block diagram is as follows: Figure 4 shown.

[0146] Further, in step S4, the suspicious lymph node identification processing steps based on sequence adaptive feature fusion include:

[0147] (Multiple sequence feature maps are fused into one feature map through feature extraction convolutional network, and voxel information, continuous change information between sequences, similarity and other features are extracted)

[0148] S41. Comprehensively classify and judge the fused sequence features and feature information of multi-frame data to obtain the location and information of suspicious lymph nodes in the feature map;

[0149] S42. Map the information such as the location of the lymph nodes in the feature map back to the original CT image, so as to locate the suspicious lymph nodes in the original image. The structure diagram of the suspicious lymph node recognition network based on sequence fusion is shown in the figure. Figure 5 shown.

[0150] Furthermore, in step S6, the comprehensive diagnosis method integrates multiple information such as rectal lesions, lymph node metastasis, lesion characteristics, etc., and combines the medical pathology information base and the TN staging prior knowledge base to make a judgment in the following steps:

[0151] S61. normalizing the suspicious lesion location information and suspicious lymph node information extracted by the feature extraction convolutional network to achieve consistency in data format;

[0152] S62. Eliminate semantic differences between different information through corresponding fusion networks, realize feature splicing and fusion, and fuse the diseased lymph node identification information and (lesion) appearance feature information;

[0153] S63. Combined with the TN staging prior knowledge base, the fused comprehensive information is classified and judged accordingly to achieve auxiliary diagnosis of clinical TN staging of rectal cancer;

[0154] S64. Map the diagnosis results back to the original CT image to achieve the corresponding TN staging annotation and visualization. The final visualization annotation is as follows Figure 6 shown.

[0155] Specifically, the overall network framework is composed of three-stage sub-network training.

[0156] The first sub-network is the training of the rectal suspected lesion area discrimination network. For this network, the segmentation network is trained, and the tumor lesion area discrimination network is trained using the enhanced CT image training sample set in the image library. The number of training samples each time is trained with batch sizes of 16, 32, 64, 128, 256, etc., and the parameters with the best training results are intercepted as the final hyperparameters. The data set is divided into three parts: training set, validation set and test set, and the specific division ratio is 6:2:2. During the training process, as the number of training iterations increases, the accuracy of the validation set continues to improve, but the test set shows a trend of first rising and then falling, that is, the network has an overfitting trend. Analyze the performance of the test set, and fine-tune the network up and down by 3% to obtain the best performance point for the diagnosis of suspected lesions. And save its parameters and network weights, and the training of the first sub-network is completed.

[0157] The second sub-network is a suspicious lymph node recognition network based on sequence adaptive features. The network was initially trained with single-frame suspicious lymph node data. The number of training samples each time was trained with batch sizes of 16, 32, 64, 128, 256, etc., and the parameters with the best training results were intercepted as the final hyperparameters. The data set division ratio is the same as that of the first sub-network division form, and the training form is similar. After obtaining the single-frame image training model results, the network structure is further improved, and multi-frame sequence information is integrated for training. The final experimental accuracy is 3% higher than the accuracy of the network using single-frame information. The parameters are further fine-tuned to obtain better performance.

[0158] The third network framework jointly trained the above two sub-networks, and through operations such as the fully connected layer, used the clinical medical prior knowledge base to perform comprehensive staging diagnosis of rectal cancer. After the joint training, the training set data was further used to fine-tune the overall network framework, and the recognition accuracy was improved by 2%.

[0159] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A deep learning-based enhanced CT image rectal cancer staging auxiliary diagnosis system, characterized by: It includes enhanced CT image input module, enhanced CT image and annotation database, image preprocessing module, rectal lesion area discrimination module, lesion lymph node recognition module, lesion feature extraction module, comprehensive diagnosis module, and visualization module; The enhanced CT image input module is used to input the rectal enhanced CT image; The enhanced CT image and annotation database is used to store the rectal enhanced CT images input by the enhanced CT image input module, the enhanced CT rectal cancer image data set and the annotation data set; The image preprocessing module is used to perform noise reduction and image enhancement processing on the enhanced CT image; The rectal lesion area discrimination module adopts a self-attention deep learning model to construct the global dependency of CT image features using the self-attention mechanism, captures the overall internal correlation of image data and image features, and is used to discriminate suspected rectal tumor areas and segment the area. The identification of the diseased lymph node recognition module adopts a method based on sequence adaptive feature fusion, using continuous multi-frame image annotation information to identify small and weak lymph node targets, and is used to identify periintestinal diseased lymph nodes; The lesion feature extraction module is used to identify and collect rectal tumor lesion area feature data, and count the number of periintestinal lesion lymph nodes to form a rectal lesion feature parameter set; The comprehensive diagnosis module is used to integrate information on rectal lesions, lymph node metastasis, and lesion characteristics, and make a judgment by comparing data from the medical pathology knowledge base; The visualization module is used to display the input enhanced CT image and mark the information related to the characteristics of the rectal tumor lesion area on the image.

2. The deep learning-based enhanced CT image rectal cancer staging auxiliary diagnosis system according to claim 1 is characterized by: The lesion feature extraction module identifies and collects characteristic parameters of the rectal tumor lesion area, including: the wall thickness of the rectal tumor lesion area, the density difference between the tumor and the normal wall, whether there are burr-like protrusions, and the involvement of adjacent structures.

3. The deep learning-based enhanced CT image rectal cancer staging auxiliary diagnosis system according to claim 2 is characterized by: The visualization module is used to display information related to the regional features of rectal tumor lesions marked on the image, including the concentrated location of rectal cancer, the thickness of the wall at the lesion, the burr protrusions around the lesion, the involvement of adjacent structures, the diseased lymph node information and the TN staging results.

4. The deep learning-based enhanced CT image rectal cancer staging auxiliary diagnosis system according to claim 2 or 3, characterized in that: The operation of the diagnostic system includes the following steps: S1. Construct enhanced CT rectal cancer image dataset and annotation database; S2. Perform format conversion and image noise reduction on enhanced CT images; S3. The rectal lesion area discrimination module uses the training data in the enhanced CT rectal cancer image dataset and the annotation database to train a self-attention-based deep learning model to determine whether the input enhanced CT image contains a suspected rectal tumor shape area and segment the lesion area; S4. The diseased lymph node recognition module uses a sequence adaptive feature fusion method to identify periintestinal diseased lymph nodes based on CT image lymph node features; S5. The lesion feature extraction module extracts the wall thickness of the rectal tumor lesion area, the density difference between the tumor and the normal wall, the burr-like protrusions around the lesion, and the appearance characteristics of the affected adjacent structures, and counts the number of peri-intestinal lesion lymph nodes to form a set of rectal lesion feature parameters; S6. The comprehensive diagnosis module integrates the rectal tumor lesion area discrimination information, diseased lymph node identification information, and lesion feature information, and combines it with the TN staging prior knowledge base to achieve clinical TN staging auxiliary diagnosis of rectal cancer.

5. The deep learning-based enhanced CT image rectal cancer staging auxiliary diagnosis system according to claim 4 is characterized by: In step S1, the steps of constructing the enhanced CT rectal cancer image dataset and annotation database include: S11. Specify a data annotation format to assign different labels to rectal cancer lesions and perianal suspicious and normal lymph nodes corresponding to different rectal cancer stages; S12. Use image annotation tools, combine the pathology knowledge base and enhanced CT image features, and annotate the enhanced CT images in accordance with the prescribed data annotation format; S13. Store the annotation file and the original image file in correspondence with each other, store the corresponding object information, and construct a data set.

6. The deep learning-based enhanced CT image rectal cancer staging auxiliary diagnosis system according to claim 4 is characterized by: In step S2, image data preprocessing is performed, and the steps include: S21. The enhanced CT rectal cancer image data stored in the original medical image in the form of dicom image series is uniformly converted into NiFTI data format, so that the original CT image data format is consistent with the corresponding annotated label data format; S22. Use the medical image processing library based on the deep learning framework to read the converted NiFTI data, convert the CT rectal cancer image data and the annotated label data into a tensor data structure processed by the deep learning framework, and establish a mapping relationship between the original image data and the label data; S23. Perform image resampling, noise reduction, random affine transformation, and image channel number addition operations on the image data and label data that establish and map the relationship, so as to achieve image enhancement.

7. The deep learning-based enhanced CT image rectal cancer staging auxiliary diagnosis system according to claim 4 is characterized by: In step S3, the rectal lesion area discrimination module uses the self-attention mechanism to construct the global dependency of CT image features and capture the overall internal correlation of image data and image features; wherein the suspected lesion area discrimination steps based on the self-attention mechanism are: S31. First, the preprocessed CT image is input into the feature extraction network, and a feature map of the corresponding depth level is obtained through one or more downsampling and pooling operations; S32. Introduce the attention mechanism, compress each channel of the feature map accordingly, pass it through the fully connected layer, and use the activation function to obtain the importance of different channels, and convert it into an attention vector; S33. Integrate the obtained feature importance weights into the original deep learning network structure feature map, further guide the focus of the network, and realize the integration of the attention mechanism; The specific formula is as follows: A=Att(X, θ)=δ(W2δ(W1GAP(X))), (1) Y=AX, (2) Among them, A is the corresponding dimension weight obtained after attention calculation; Att() is the attention calculation operation function, and X is the feature map data extracted by the feature extraction convolutional network; θ is the network parameter, δ is the ReLu activation function, which is used to provide nonlinear gating operations in the network; W1 and W2 correspond to two fully connected layers to achieve feature dimensionality reduction, and GAP() is the global average pooling operation function; Y is the feature of feature map X after calculation by the attention module.

8. The deep learning-based enhanced CT image rectal cancer staging auxiliary diagnosis system according to claim 4 is characterized by: In step S4, the lymph node diagnosis processing steps based on sequence adaptive feature fusion include: S41. Comprehensively classify and judge the fused sequence features and feature information of multi-frame data to obtain the location and information of suspicious lymph nodes in the feature map; S42. Map the lymph node location information in the feature map back to the original enhanced CT image.

9. The deep learning-based enhanced CT image rectal cancer staging auxiliary diagnosis system according to claim 4 is characterized by: In step S6, the comprehensive diagnosis module is used to integrate the rectal lesion area discrimination information, the lesion lymph node identification information, and the characteristics of the inspection object, and compare with the TN staging prior knowledge base to give the TN staging auxiliary diagnosis result of rectal cancer: S61. normalizing the suspicious lesion location information, suspicious lymph nodes, and appearance feature information extracted by the feature extraction network to achieve consistency in data format; S62. Eliminate semantic differences between different information through corresponding fusion networks, and realize splicing and fusion of features; S63. According to the comparison with the corresponding clinical knowledge base, the integrated information is classified and judged accordingly to achieve a comprehensive judgment of the information on rectal cancer; S64. Map the judgment result of S63 back to the original enhanced CT image to achieve corresponding TN labeling and visualization.

10. The deep learning-based enhanced CT image rectal cancer staging auxiliary diagnosis system according to claim 5, characterized in that: In step S11, four different labels are specified for the intestinal wall, and the correspondence between the stages and the labels is as follows: T0-label1, T1-label2, T2-label3, T3-label4, T4-label5; two different labels are specified for the lymph nodes, and the correspondence between the status and the label is: normal lymph nodes-label6, suspicious lymph nodes-label7.

Images