An image-based auxiliary diagnosis method and system for rectal cancer

Abstract

The present invention relates to the field of image processing technology, and particularly relates to an image-based auxiliary diagnosis method and system for rectal cancer. An image-based auxiliary diagnosis system for rectal cancer is provided, including: a microscopic examination picture acquisition module and a rectal cancer analysis module. The present invention analyzes the microscopic examination pictures of patients through a rectal cancer analysis model, and in the rectal cancer analysis model, the feature maps are enhanced by the interaction of the feature fields of different blocks in the first standard feature map, the second standard feature map, the third standard feature map, and the fourth standard feature map through a feature collaboration network, which can improve the feature expression ability and global information expression ability of the feature maps. Further enhancement is also carried out on the first fusion feature map, the second fusion feature map, the third fusion feature map, and the fourth fusion feature map through the focus degrees of different branches, enhancing the versatility of the rectal cancer analysis model and improving the analysis accuracy of the rectal cancer analysis model.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly relates to a method and system for assisting in the diagnosis of rectal cancer based on images. Background Art

[0002] In the field of medical image diagnosis, the early detection and diagnosis of rectal cancer are of great significance for improving the survival rate of patients. However, traditional methods for diagnosing rectal cancer mainly rely on doctors' manual observation and analysis of medical images (such as endoscopic images, pathological section images, etc.). Although this manual diagnosis method can provide reliable diagnosis results to a certain extent, it also has significant limitations. First of all, doctors need to process a large amount of medical image data for a long time and with high intensity, especially in the screening stage, which greatly increases the workload of doctors. With the continuous growth of medical needs and the relative shortage of high-quality medical resources, the diagnostic workload of doctors is increasing day by day, which is likely to lead to fatigue and decreased attention, thus possibly affecting the accuracy of diagnosis. Summary of the Invention

[0003] The present invention analyzes the endoscopic pictures of patients through a rectal cancer analysis model to assist in the diagnosis of rectal cancer, which can reduce the workload of doctors. And in the rectal cancer analysis model, the feature maps are enhanced by the interaction of the feature fields of different blocks in the first standard feature map, the second standard feature map, the third standard feature map, and the fourth standard feature map through a feature collaboration network, which can improve the feature expression ability and global information expression ability of the feature maps. Further enhancement is also carried out on the first fusion feature map, the second fusion feature map, the third fusion feature map, and the fourth fusion feature map through the degree of focus of different branches, which enhances the versatility of the rectal cancer analysis model and improves the analysis accuracy of the rectal cancer analysis model.

[0004] The invention provides a method for assisting in the diagnosis of rectal cancer based on images, including:

[0005] Obtaining the endoscopic pictures of the patient, and then sending the endoscopic pictures of the patient into the rectal cancer analysis model for processing to output a rectal cancer analysis label;

[0006] The rectal cancer analysis model includes a branch feature extraction network, a feature alignment network, a feature collaboration network, a dynamic branch activation network, a feature competition network, and a decision analysis network. The branch feature extraction network includes a first residual block, a second residual block, a third residual block, and a fourth residual block. The branch feature extraction network is used to perform feature extraction operations on microscopic examination pictures, and the feature maps output by the first residual block, the second residual block, the third residual block, and the fourth residual block are respectively denoted as the first feature map, the second feature map, the third feature map, and the fourth feature map. The feature alignment network is used to perform feature alignment operations on the first feature map, the second feature map, the third feature map, and the fourth feature map to construct corresponding first standard feature maps, second standard feature maps, third standard feature maps, and fourth standard feature maps. The feature collaboration network is used to perform block operations on the first standard feature map, the second standard feature map, the third standard feature map, and the fourth standard feature map respectively, then construct feature fields for the block-processed first standard feature map, second standard feature map, third standard feature map, and fourth standard feature map respectively, and generate corresponding feature collaboration matrices. Based on the feature collaboration matrices, feature enhancement is performed on the corresponding first standard feature map, second standard feature map, third standard feature map, and fourth standard feature map to construct corresponding first fusion feature maps, second fusion feature maps, third fusion feature maps, and fourth fusion feature maps. The dynamic branch activation network is used to process the first feature map, the second feature map, the third feature map, and the fourth feature map respectively through a first multi-layer perceptron, and output the activation weights corresponding to the first feature map, the second feature map, the third feature map, and the fourth feature map. The feature competition network is used to select the first fusion feature map, the second fusion feature map, the third fusion feature map, and the fourth fusion feature map for self-attention operations based on the activation weights corresponding to the first feature map, the second feature map, the third feature map, and the fourth feature map, and update the first fusion feature map, the second fusion feature map, the third fusion feature map, and the fourth fusion feature map to construct corresponding first enhanced feature maps, second enhanced feature maps, third enhanced feature maps, and fourth enhanced feature maps. The decision analysis network is used to perform weighted fusion operations on the first enhanced feature map, the second enhanced feature map, the third enhanced feature map, and the fourth enhanced feature map based on the activation weights corresponding to the first feature map, the second feature map, the third feature map, and the fourth feature map, that is, multiply the first enhanced feature map, the second enhanced feature map, the third enhanced feature map, and the fourth enhanced feature map by the corresponding activation weights and then splice them according to channels, and then perform convolution and fully connected operations, and generate rectal cancer analysis labels.

[0007] Preferably, the feature alignment network performs feature alignment operations on the first feature map, the second feature map, the third feature map, and the fourth feature map to construct corresponding first standard feature maps, second standard feature maps, third standard feature maps, and fourth standard feature maps, specifically including the following steps:

[0008] Traverse the first feature map, the second feature map, the third feature map, and the fourth feature map, and denote the selected feature map as the target feature map. Then, perform a one-dimensional unfolding on the target feature map to construct a target feature vector. Next, perform a position encoding operation on the target feature map to construct a position vector. Execute an addition operation on the target feature vector and the position vector to construct an intermediate feature vector. Send the intermediate feature vector into a second multi-layer perceptron for processing to obtain an output feature vector. Then, fold the output feature vector into a first output feature map, and the size of the first output feature map is the same as that of the microscopy image; the first output feature map is the first standard feature map, the second standard feature map, the third standard feature map, and the fourth standard feature map corresponding to the selected feature map.

[0009] Preferably, perform a block operation on the first standard feature map, the second standard feature map, the third standard feature map, and the fourth standard feature map respectively through a feature collaboration network. Then, construct a feature field for the block-operated first standard feature map, second standard feature map, third standard feature map, and fourth standard feature map respectively, and generate corresponding feature collaboration matrices. Based on the feature collaboration matrices, perform feature enhancement on the corresponding first standard feature map, second standard feature map, third standard feature map, and fourth standard feature map to construct corresponding first fusion feature maps, second fusion feature maps, third fusion feature maps, and fourth fusion feature maps. The specific steps are as follows:

[0010] Traverse the first standard feature map, the second standard feature map, the third standard feature map, and the fourth standard feature map. Denote the selected standard feature map as the target standard feature map. Perform a block operation on the target standard feature map through a sliding window with a step size of 1 to construct a number of local feature maps. Denote the central coordinate position of the i-th local feature map as the feature block position W i , i = 1, 2, 3, …, I, where I is the total number of local feature maps. Select the maximum value in the i-th local feature map and denote it as the feature block intensity Q i , and denote the gradient direction calculated for the i-th local feature map through the Sobel operator as the feature block direction F i , the feature block positions W i , feature block intensities Q i , and feature block directions F i corresponding to all local feature maps form a feature field. For each local feature map, calculate the feature collaboration weight corresponding to the i-th local feature map , where P i (t) is the feature collaboration weight corresponding to the i -th local feature map after the t-th iteration, k = 1, 2, 3, …, I, and k ≠ i , that is, select all local feature maps except the i-th local feature map. μ is the feature field intensity coefficient, |W i , W k| is the distance between the i-th local feature map and the k-th local feature map, and cos() is the cosine calculation; until the number of iterations reaches the maximum number of iterations, the feature collaboration weights corresponding to all local feature maps are composed into a feature collaboration matrix according to the positions of the local feature maps, and the target standard feature map is multiplied by the corresponding feature collaboration matrix to construct the second output feature map corresponding to the target standard feature map. The second output feature map is the first fusion feature map, the second fusion feature map, the third fusion feature map, and the fourth fusion feature map corresponding to the selected standard feature map.

[0011] Preferably, a feature competition network is used to select the first fusion feature map, the second fusion feature map, the third fusion feature map, and the fourth fusion feature map for self-attention operation based on the activation weights corresponding to the first feature map, the second feature map, the third feature map, and the fourth feature map, and update the first fusion feature map, the second fusion feature map, the third fusion feature map, and the fourth fusion feature map to construct the corresponding first enhanced feature map, second enhanced feature map, third enhanced feature map, and fourth enhanced feature map. The specific steps are as follows:

[0012] Select the first feature map, the second feature map, the third feature map, or the fourth feature map corresponding to the maximum activation weight and denote it as the target activation feature map;

[0013] Traverse the first fusion feature map, the second fusion feature map, the third fusion feature map, and the fourth fusion feature map, denote the selected fusion feature map as the target fusion feature map, multiply the target fusion feature map by the value weight matrix and the key weight matrix to construct the target value matrix A and the target key matrix B, multiply the target activation feature map by the query weight matrix to construct the target query matrix H, and calculate the self-attention weight matrix ATT = softmax(HB T / (d) 0.5 ), where d is the dimension size of the target fusion feature map, and multiply the target value matrix A by the self-attention weight matrix ATT to construct the third output feature map. The third output feature map is the first enhanced feature map, the second enhanced feature map, the third enhanced feature map, and the fourth enhanced feature map corresponding to the selected fusion feature map.

[0014] Preferably, the rectal cancer analysis model is trained, and the specific steps are as follows:

[0015] Obtain a number of rectal cancer analysis training samples, where the rectal cancer analysis training samples are microscopic examination pictures, and label the rectal cancer analysis training samples with rectal cancer analysis labels, and form a rectal cancer analysis training set with the labeled rectal cancer analysis training samples. Train the rectal cancer analysis model with the rectal cancer analysis training set, and determine whether the training conditions are met. If the training conditions are met, output the trained rectal cancer analysis model; otherwise, train the rectal cancer analysis model with the rectal cancer analysis training set.

[0016] Preferably, the hyperparameters of the rectal cancer analysis model are simulated and set through a swarm optimization algorithm, and the specific steps are as follows:

[0017] Construct a number of hyperparameter simulation individuals; form a population set with all hyperparameter simulation individuals, and set the maximum number of iterations;

[0018] Calculate the fitness corresponding to the hyperparameter simulation individuals;

[0019] Based on the fitness of the hyperparameter simulation individuals, iteratively update the population set through a swarm optimization algorithm;

[0020] Until the number of iterations reaches the maximum number of iterations, output the hyperparameter simulation individual with the maximum fitness to perform hyperparameter setting on the rectal cancer analysis training set.

[0021] Preferably, the calculation method of the fitness corresponding to the hyperparameter simulation individuals is: perform hyperparameter setting on the rectal cancer analysis model through the hyperparameter simulation individuals, and then train the rectal cancer analysis model with the set hyperparameters using the rectal cancer analysis training set, and use the accuracy of the rectal cancer analysis model as the fitness corresponding to the hyperparameter simulation individuals.

[0022] The present invention also provides an image-based rectal cancer auxiliary diagnosis system, including:

[0023] A microscopic examination picture acquisition module for acquiring microscopic examination pictures of patients;

[0024] A rectal cancer analysis module for sending the microscopic examination pictures of patients into the rectal cancer analysis model for processing and outputting rectal cancer analysis labels;

[0025] The rectal cancer analysis model includes a branch feature extraction network, a feature alignment network, a feature collaboration network, a dynamic branch activation network, a feature competition network, and a decision analysis network. The branch feature extraction network includes a first residual block, a second residual block, a third residual block, and a fourth residual block. The branch feature extraction network is used to perform feature extraction operations on microscopic examination pictures, and the feature maps output by the first residual block, the second residual block, the third residual block, and the fourth residual block are respectively denoted as the first feature map, the second feature map, the third feature map, and the fourth feature map. The feature alignment network is used to perform feature alignment operations on the first feature map, the second feature map, the third feature map, and the fourth feature map to construct corresponding first standard feature maps, second standard feature maps, third standard feature maps, and fourth standard feature maps. The feature collaboration network is used to perform block operations on the first standard feature map, the second standard feature map, the third standard feature map, and the fourth standard feature map respectively, then construct feature fields for the first standard feature map, the second standard feature map, the third standard feature map, and the fourth standard feature map after block operations respectively, and generate corresponding feature collaboration matrices. Based on the feature collaboration matrices, perform feature enhancement on the corresponding first standard feature map, second standard feature map, third standard feature map, and fourth standard feature map to construct corresponding first fusion feature maps, second fusion feature maps, third fusion feature maps, and fourth fusion feature maps. The dynamic branch activation network is used to process the first feature map, the second feature map, the third feature map, and the fourth feature map respectively through a first multi-layer perceptron, and output the activation weights corresponding to the first feature map, the second feature map, the third feature map, and the fourth feature map. The feature competition network is used to select the first fusion feature map, the second fusion feature map, the third fusion feature map, and the fourth fusion feature map for self-attention operations based on the activation weights corresponding to the first feature map, the second feature map, the third feature map, and the fourth feature map, and update the first fusion feature map, the second fusion feature map, the third fusion feature map, and the fourth fusion feature map to construct corresponding first enhanced feature maps, second enhanced feature maps, third enhanced feature maps, and fourth enhanced feature maps. The decision analysis network is used to perform weighted fusion operations on the first enhanced feature map, the second enhanced feature map, the third enhanced feature map, and the fourth enhanced feature map based on the activation weights corresponding to the first feature map, the second feature map, the third feature map, and the fourth feature map, and generate rectal cancer analysis labels.

[0026] The present invention has the following advantages:

[0027] The present invention analyzes the endoscopic images of patients through a rectal cancer analysis model to assist in the diagnosis of rectal cancer, which can reduce the workload of doctors. In the rectal cancer analysis model, a feature collaborative network is used to enhance the feature map by interacting with the feature fields of different blocks in the first standard feature map, the second standard feature map, the third standard feature map, and the fourth standard feature map, which can improve the feature expression ability and global information expression ability of the feature map. Further enhancement is also performed on the first fusion feature map, the second fusion feature map, the third fusion feature map, and the fourth fusion feature map through the focus degrees of different branches, enhancing the versatility of the rectal cancer analysis model and improving the analysis accuracy of the rectal cancer analysis model. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 FIG. is a schematic structural diagram of the rectal cancer analysis model adopted in the embodiment of the present invention.

[0029] Figure 2 FIG. is a schematic structural diagram of the image-based rectal cancer auxiliary diagnosis system adopted in the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0030] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0031] Embodiment 1, an image-based rectal cancer auxiliary diagnosis method, includes:

[0032] Obtain the endoscopic images of the patient. Here, the endoscopic images refer to the images taken of the surface of the patient's rectal mucosa through narrow-band imaging endoscopy. The characteristics of the rectal blood vessels and the mucosal surface in the endoscopic images can assist in the diagnosis of rectal cancer. Then, send the endoscopic images of the patient into the rectal cancer analysis model for processing, and output the rectal cancer analysis label. The rectal cancer analysis label generally includes benign, malignant, and normal, etc. And malignant means suffering from rectal cancer, while benign generally refers to rectal lesions such as rectal polyps or rectal adenomas. Assisting in the diagnosis of rectal cancer through the rectal cancer analysis model can reduce the workload of doctors;

[0033] See Figure 1, the rectal cancer analysis model includes a branch feature extraction network, a feature alignment network, a feature collaboration network, a dynamic branch activation network, a feature competition network, and a decision analysis network. Among them, the branch feature extraction network is established based on the ResNet50 model and includes a first residual block, a second residual block, a third residual block, and a fourth residual block. The branch feature extraction network is used to perform feature extraction operations on microscopic examination pictures, and the feature maps output by the first residual block, the second residual block, the third residual block, and the fourth residual block are respectively denoted as the first feature map, the second feature map, the third feature map, and the fourth feature map. It should be noted that the operations of the first residual block, the second residual block, the third residual block, and the fourth residual block are regarded as feature extraction branches at different depths. The first feature map corresponds to low-depth features, such as edges and color gradients, etc.; the second feature map corresponds to relatively low-depth features, such as texture features, etc.; the third feature map corresponds to medium-depth features, such as local structure features; the fourth feature map corresponds to high-depth features, such as global semantic features. And the first residual block, the second residual block, the third residual block, and the fourth residual block are set with reference to the ResNet50 model and all include multi-level convolutional operations; the feature alignment network is used to perform feature alignment operations on the first feature map, the second feature map, the third feature map, and the fourth feature map to construct corresponding first standard feature maps, second standard feature maps, third standard feature maps, and fourth standard feature maps; the feature collaboration network is used to perform block operations on the first standard feature map, the second standard feature map, the third standard feature map, and the fourth standard feature map respectively, then perform feature field construction on the blocked first standard feature map, second standard feature map, third standard feature map, and fourth standard feature map respectively, and generate corresponding feature collaboration matrices. Based on the feature collaboration matrices, perform feature enhancement on the corresponding first standard feature map, second standard feature map, third standard feature map, and fourth standard feature map to construct corresponding first fusion feature maps, second fusion feature maps, third fusion feature maps, and fourth fusion feature maps. It should be noted that performing feature field construction on the blocked first standard feature map, second standard feature map, third standard feature map, and fourth standard feature map respectively can learn the interaction strength and direction between different blocks in the feature map, and perform feature fusion through feature field interaction between different blocks, which can achieve higher-quality feature maps;The dynamic branch activation network is used to process the first feature map, the second feature map, the third feature map, and the fourth feature map respectively by feeding them into the first multi-layer perceptron, and output the activation weights corresponding to the first feature map, the second feature map, the third feature map, and the fourth feature map. The activation weights represent the degree of focus on different branch features in the microscopy image. For example, for a small lesion image, it focuses on the branch of tiny features, and for a large lesion image, it focuses on the branch of global context features. And there may be problems such as noise, artifacts, and uneven contrast in the microscopy image. By activating different branch features, the robustness of the rectal cancer analysis model in the face of these problems can be enhanced. For example, if the features extracted by a certain branch are interfered by noise, the dynamic branch mechanism can weaken the role of this branch and instead rely on the features extracted by other branches; The feature competition network is used to select the first fused feature map, the second fused feature map, the third fused feature map, and the fourth fused feature map for self-attention operation based on the activation weights corresponding to the first feature map, the second feature map, the third feature map, and the fourth feature map, and update the first fused feature map, the second fused feature map, the third fused feature map, and the fourth fused feature map to construct the corresponding first enhanced feature map, second enhanced feature map, third enhanced feature map, and fourth enhanced feature map; The decision analysis network is used to perform a weighted fusion operation on the first enhanced feature map, the second enhanced feature map, the third enhanced feature map, and the fourth enhanced feature map based on the activation weights corresponding to the first feature map, the second feature map, the third feature map, and the fourth feature map, and generate a rectal cancer analysis label;

[0034] This application uses a rectal cancer analysis model to analyze the microscopy images of patients and assist in the diagnosis of rectal cancer, which can reduce the workload of doctors. And in the rectal cancer analysis model, the feature collaborative network is used to strengthen the feature map by interacting the feature fields of different blocks in the first standard feature map, the second standard feature map, the third standard feature map, and the fourth standard feature map, which can improve the feature expression ability and global information expression ability of the feature map. It also further strengthens the first fused feature map, the second fused feature map, the third fused feature map, and the fourth fused feature map through the degree of focus of different branches, enhances the versatility of the rectal cancer analysis model, and improves the analysis accuracy of the rectal cancer analysis model.

[0035] The feature alignment network performs feature alignment operations on the first feature map, the second feature map, the third feature map, and the fourth feature map to construct the corresponding first standard feature map, second standard feature map, third standard feature map, and fourth standard feature map. The specific steps are as follows:

[0036] Traverse the first feature map, the second feature map, the third feature map, and the fourth feature map, and denote the selected feature map as the target feature map. Then, perform a one-dimensional unfolding on the target feature map to construct a target feature vector. Next, perform a position encoding operation on the target feature map to construct a position vector. The position encoding operation refers to the position encoding of the Transformer model and uses sine and cosine functions. Through the position encoding operation, the spatial structure in the feature map can be retained. Perform an addition operation on the target feature vector and the position vector to construct an intermediate feature vector. Send the intermediate feature vector into a second multi-layer perceptron for processing to obtain an output feature vector. Then, fold the output feature vector into a first output feature map. Here, folding means restoring the output feature vector to a three-dimensional data structure, and the size of the first output feature map is the same as that of the microscopy image; the first output feature map is the first standard feature map, the second standard feature map, the third standard feature map, and the fourth standard feature map corresponding to the selected feature map.

[0037] Perform a block operation on the first standard feature map, the second standard feature map, the third standard feature map, and the fourth standard feature map respectively through a feature collaboration network. Then, construct a feature field for each of the block-processed first standard feature map, second standard feature map, third standard feature map, and fourth standard feature map, and generate corresponding feature collaboration matrices. Based on the feature collaboration matrices, perform feature enhancement on the corresponding first standard feature map, second standard feature map, third standard feature map, and fourth standard feature map to construct corresponding first fusion feature maps, second fusion feature maps, third fusion feature maps, and fourth fusion feature maps. The specific steps are as follows:

[0038] Traverse the first standard feature map, the second standard feature map, the third standard feature map, and the fourth standard feature map, and denote the selected standard feature map as the target standard feature map. Perform a block operation on the target standard feature map through a sliding window with a step size of 1 to construct a number of local feature maps. Generally, the size of the sliding window is 3×3. Denote the central coordinate position of the i-th local feature map as the feature block position W i , where i = 1, 2, 3, …, I, and I is the total number of local feature maps. Select the maximum value in the i-th local feature map and denote it as the feature block intensity Q i , and denote the gradient direction calculated for the i-th local feature map through the Sobel operator as the feature block direction F i , and all the feature block positions W i , feature block intensities Q i and feature block directions F i of the local feature maps form a feature field. For each local feature map, calculate the feature collaboration weight corresponding to the i-th local feature map , where P i (t) is the iThe feature collaboration weights corresponding to each local feature map, k = 1, 2, 3, …, I, and k ≠ i , that is, all local feature maps except the i-th local feature map are selected. μ is the feature field intensity coefficient, which is set manually. |W i , W k | is the distance between the i-th local feature map and the k-th local feature map, and cos() is the cosine calculation; until the number of iterations reaches the maximum number of iterations, the feature collaboration weights corresponding to all local feature maps are composed into a feature collaboration matrix according to the positions of the local feature maps, and the target standard feature map is multiplied by the corresponding feature collaboration matrix to construct the second output feature map corresponding to the target standard feature map. The second output feature map is the first fusion feature map, the second fusion feature map, the third fusion feature map, and the fourth fusion feature map corresponding to the selected standard feature map.

[0039] The feature competition network is used to select the first fusion feature map, the second fusion feature map, the third fusion feature map, and the fourth fusion feature map based on the activation weights corresponding to the first feature map, the second feature map, the third feature map, and the fourth feature map for self-attention operation, and update the first fusion feature map, the second fusion feature map, the third fusion feature map, and the fourth fusion feature map to construct the corresponding first enhanced feature map, second enhanced feature map, third enhanced feature map, and fourth enhanced feature map. The specific steps are as follows:

[0040] Select the first feature map, the second feature map, the third feature map, or the fourth feature map corresponding to the largest activation weight and denote it as the target activation feature map;

[0041] Traverse the first fusion feature map, the second fusion feature map, the third fusion feature map, and the fourth fusion feature map, denote the selected fusion feature map as the target fusion feature map, multiply the target fusion feature map by the value weight matrix and the key weight matrix to construct the target value matrix A and the target key matrix B, multiply the target activation feature map by the query weight matrix to construct the target query matrix H, and calculate the self-attention weight matrix ATT = softmax(HB T / (d) 0.5 ), where d is the dimension size of the target fusion feature map, and multiply the target value matrix A by the self-attention weight matrix ATT to construct the third output feature map. The third output feature map is the first enhanced feature map, the second enhanced feature map, the third enhanced feature map, and the fourth enhanced feature map corresponding to the selected fusion feature map; the value weight matrix, the key weight matrix, and the query weight matrix here refer to the settings of the self-attention mechanism in the Transformer model.

[0042] Train the rectal cancer analysis model. The specific steps are as follows:

[0043] Obtain several rectal cancer analysis training samples. The rectal cancer analysis training samples are microscopic examination pictures actually collected by operators, and the rectal cancer analysis training samples are labeled with rectal cancer analysis labels. The labeled rectal cancer analysis training samples are composed into a rectal cancer analysis training set, and the rectal cancer analysis model is trained through the rectal cancer analysis training set to determine whether the training conditions are met. The training conditions generally mean that the accuracy rate of the rectal cancer analysis model meets the expectations. If the training conditions are met, the trained rectal cancer analysis model is output; otherwise, the rectal cancer analysis model is trained through the rectal cancer analysis training set.

[0044] In order to further improve the accuracy rate of the rectal cancer analysis model, it also includes simulating and setting the hyperparameters of the rectal cancer analysis model through a swarm optimization algorithm. The specific steps are as follows:

[0045] Construct several hyperparameter simulation individuals. The hyperparameter simulation individuals include the hyperparameters in the rectal cancer analysis model, such as the maximum training iteration times and the learning rate, etc. The way to construct the hyperparameter simulation individuals is to assign random values to the hyperparameters within the value range of the hyperparameters; form a population set with all the hyperparameter simulation individuals and set the maximum iteration times;

[0046] Calculate the fitness corresponding to the hyperparameter simulation individuals. The calculation method is: set the hyperparameters of the rectal cancer analysis model through the hyperparameter simulation individuals, and then train the rectal cancer analysis model with the set hyperparameters through the rectal cancer analysis training set, and use the accuracy rate of the rectal cancer analysis model as the fitness corresponding to the hyperparameter simulation individuals;

[0047] Based on the fitness of the hyperparameter simulation individuals, iteratively update the population set through the sparrow search algorithm;

[0048] Until the iteration times reach the maximum iteration times, output the hyperparameter simulation individual with the maximum fitness to set the hyperparameters for the rectal cancer analysis training set.

[0049] Embodiment 2, an image-based rectal cancer auxiliary diagnosis system, as Figure 2 shown, includes:

[0050] A microscopic examination picture acquisition module, used to acquire the microscopic examination pictures of patients. Here, the microscopic examination pictures refer to the pictures taken of the rectal mucosa surface of patients through narrow-band imaging endoscopes. The characteristics of the rectal blood vessels and the mucosa surface in the microscopic examination pictures can assist in the diagnosis of rectal cancer;

[0051] The rectal cancer analysis module is used to send the endoscopic examination pictures of patients into the rectal cancer analysis model for processing and output the rectal cancer analysis labels. The rectal cancer analysis labels generally include benign, malignant, and normal, etc. Malignant means suffering from rectal cancer, while benign generally refers to rectal lesions such as rectal polyps or rectal adenomas. Assisting in the diagnosis of rectal cancer through the rectal cancer analysis model can reduce the workload of doctors;

[0052] The rectal cancer analysis model includes a branch feature extraction network, a feature alignment network, a feature collaboration network, a dynamic branch activation network, a feature competition network, and a decision analysis network. Among them, the branch feature extraction network is established based on the ResNet50 model and includes a first residual block, a second residual block, a third residual block, and a fourth residual block. The branch feature extraction network is used to perform feature extraction operations on microscopic examination pictures, and the feature maps output by the first residual block, the second residual block, the third residual block, and the fourth residual block are respectively denoted as the first feature map, the second feature map, the third feature map, and the fourth feature map. It should be noted that the operations of the first residual block, the second residual block, the third residual block, and the fourth residual block are regarded as feature extraction branches at different depths. The first feature map corresponds to low-depth features, such as edges and color gradients, etc.; the second feature map corresponds to relatively low-depth features, such as texture features, etc.; the third feature map corresponds to medium-depth features, such as local structure features; the fourth feature map corresponds to high-depth features, such as global semantic features. And the first residual block, the second residual block, the third residual block, and the fourth residual block are set with reference to the ResNet50 model and all include multi-level convolutional operations. The feature alignment network is used to perform feature alignment operations on the first feature map, the second feature map, the third feature map, and the fourth feature map to construct corresponding first standard feature maps, second standard feature maps, third standard feature maps, and fourth standard feature maps. The feature collaboration network is used to perform block operations on the first standard feature map, the second standard feature map, the third standard feature map, and the fourth standard feature map respectively, and then perform feature field construction on the blocked first standard feature map, second standard feature map, third standard feature map, and fourth standard feature map respectively, and generate corresponding feature collaboration matrices. Based on the feature collaboration matrices, perform feature enhancement on the corresponding first standard feature map, second standard feature map, third standard feature map, and fourth standard feature map to construct corresponding first fusion feature maps, second fusion feature maps, third fusion feature maps, and fourth fusion feature maps. It should be noted that performing feature field construction on the blocked first standard feature map, second standard feature map, third standard feature map, and fourth standard feature map respectively can learn the interaction strength and direction between different blocks in the feature map, and perform feature fusion through feature field interaction between different blocks, which can achieve higher-quality feature maps.The dynamic branch activation network is used to process the first feature map, the second feature map, the third feature map, and the fourth feature map respectively by feeding them into the first multi-layer perceptron, and output the activation weights corresponding to the first feature map, the second feature map, the third feature map, and the fourth feature map. The activation weights represent the degree of focus on different branch features in the microscopic examination image. For example, for small lesion images, it focuses on the branch of microscopic features, and for large lesion images, it focuses on the branch of global context features. And there may be problems such as noise, artifacts, and uneven contrast in the microscopic examination image. By activating different branch features, the robustness of the rectal cancer analysis model in the face of these problems can be enhanced. For example, if the features extracted by a certain branch are interfered by noise, the dynamic branch mechanism can weaken the role of this branch and instead rely on the features extracted by other branches; The feature competition network is used to select the first fused feature map, the second fused feature map, the third fused feature map, and the fourth fused feature map for self-attention operation based on the activation weights corresponding to the first feature map, the second feature map, the third feature map, and the fourth feature map, and update the first fused feature map, the second fused feature map, the third fused feature map, and the fourth fused feature map to construct the corresponding first enhanced feature map, second enhanced feature map, third enhanced feature map, and fourth enhanced feature map; The decision analysis network is used to perform a weighted fusion operation on the first enhanced feature map, the second enhanced feature map, the third enhanced feature map, and the fourth enhanced feature map based on the activation weights corresponding to the first feature map, the second feature map, the third feature map, and the fourth feature map, and generate a rectal cancer analysis label.

[0053] It should be understood that those of ordinary skill in the art can make improvements or transformations according to the above description, and all such improvements and transformations should fall within the protection scope of the appended claims of the present invention. The parts not described in detail in this specification belong to the well-known prior art of those skilled in the art.

Claims

1. An image-based auxiliary diagnosis method for rectal cancer, characterized in that: include: Obtain the patient's microscopic images, and then send the patient's microscopic images to the rectal cancer analysis model for processing, and output the rectal cancer analysis label; The rectal cancer analysis model includes a branch feature extraction network, a feature alignment network, a feature collaboration network, a dynamic branch activation network, a feature competition network and a decision analysis network, wherein the branch feature extraction network includes a first residual block, a second residual block, a third residual block and a fourth residual block, and the branch feature extraction network is used to perform feature extraction operations on microscopic images, and the feature maps output by the first residual block, the second residual block, the third residual block and the fourth residual block are respectively recorded as the first feature map, the second feature map, the third feature map and the fourth feature map; the feature alignment network is used to perform feature alignment operations on the first feature map, the second feature map, the third feature map and the fourth feature map to construct corresponding first standard feature maps, second standard feature maps, third standard feature maps and fourth standard feature maps; The feature synergy network is used to perform block operations on the first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map, respectively, and then construct feature fields for the first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map after blocking, and generate corresponding feature synergy matrices. Based on the feature synergy matrix, the corresponding first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map are feature enhanced to construct the corresponding first fusion feature map, the second fusion feature map, the third fusion feature map and the fourth fusion feature map; the dynamic branch activation network is used to send the first feature map, the second feature map, the third feature map and the fourth feature map to the first multilayer perceptron for processing, and output the first feature map, the second feature map, the third feature map and the fourth feature map. The activation weights corresponding to the feature map and the fourth feature map; the feature competition network is used to select the first fused feature map, the second fused feature map, the third fused feature map and the fourth fused feature map for self-attention operation based on the activation weights corresponding to the first feature map, the second feature map, the third feature map and the fourth feature map, and update the first fused feature map, the second fused feature map, the third fused feature map and the fourth fused feature map to construct the corresponding first enhanced feature map, the second enhanced feature map, the third enhanced feature map and the fourth enhanced feature map; the decision analysis network is used to perform a weighted fusion operation on the first enhanced feature map, the second enhanced feature map, the third enhanced feature map and the fourth enhanced feature map based on the activation weights corresponding to the first feature map, the second feature map, the third feature map and the fourth feature map, and generate a rectal cancer analysis label; The first feature map, the second feature map, the third feature map and the fourth feature map are subjected to a feature alignment operation through a feature alignment network to construct corresponding first standard feature maps, second standard feature maps, third standard feature maps and fourth standard feature maps, specifically comprising the following steps: Traverse the first feature map, the second feature map, the third feature map and the fourth feature map, and record the selected feature map as the target feature map, then expand the target feature map in one dimension, construct a target feature vector, then perform a position encoding operation on the target feature map, construct a position vector, perform an addition operation on the target feature vector and the position vector, construct an intermediate feature vector, send the intermediate feature vector to the second multi-layer perceptron for processing, obtain an output feature vector, and then fold the output feature vector into the first output feature map, and the size of the first output feature map is consistent with the microscopic image; the first output feature map is the first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map corresponding to the selected feature map; The first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map are respectively divided into blocks through a feature collaboration network, and then the feature fields are respectively constructed for the divided first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map, and a corresponding feature collaboration matrix is ​​generated. Based on the feature collaboration matrix, the corresponding first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map are feature enhanced to construct the corresponding first fusion feature map, the second fusion feature map, the third fusion feature map and the fourth fusion feature map, which specifically includes the following steps: Traverse the first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map, record the selected standard feature map as the target standard feature map, perform a block operation on the target standard feature map through a sliding window with a step size of 1, construct several local feature maps, and record the central coordinate position of the i-th local feature map as the feature block position W i , i=1, 2, 3, ..., I, I is the total number of local feature maps, and the maximum value in the i-th local feature map is selected as the feature block strength Q i , the gradient direction calculated by the Sobel operator in the i-th local feature map is recorded as the feature block direction F i , the feature block positions W corresponding to all local feature maps i , feature block strength Q i and feature block direction F i Form a feature field, and for each local feature map, calculate the feature coordination weight corresponding to the i-th local feature map , where P i (t) is the number of i The feature coordination weights corresponding to the local feature maps, k=1, 2, 3, ..., I, and k≠ i , that is, select all local feature maps except the i-th local feature map, μ is the feature field strength coefficient, |W i , W k | is the distance between the ith local feature map and the kth local feature map, and cos() is the cosine calculation; until the number of iterations reaches the maximum number of iterations, the feature synergy weights corresponding to all local feature maps are composed of a feature synergy matrix according to the positions of the local feature maps, and the target standard feature map is multiplied by the corresponding feature synergy matrix to construct the second output feature map corresponding to the target standard feature map. The second output feature map is the first fused feature map, the second fused feature map, the third fused feature map, and the fourth fused feature map corresponding to the selected standard feature map.

2. The image-based auxiliary diagnosis method for rectal cancer according to claim 1, characterized in that: The feature competition network is used to select the first fused feature map, the second fused feature map, the third fused feature map and the fourth fused feature map for self-attention operation based on the activation weights corresponding to the first feature map, the second feature map, the third feature map and the fourth feature map, and the first fused feature map, the second fused feature map, the third fused feature map and the fourth fused feature map are updated to construct the corresponding first enhanced feature map, the second enhanced feature map, the third enhanced feature map and the fourth enhanced feature map, specifically including the following steps: Select the first feature map, the second feature map, the third feature map or the fourth feature map corresponding to the maximum activation weight as the target activation feature map; Traverse the first fusion feature map, the second fusion feature map, the third fusion feature map and the fourth fusion feature map, record the selected fusion feature map as the target fusion feature map, multiply the target fusion feature map with the value weight matrix and the key weight matrix to construct the target value matrix A and the target key matrix B, multiply the target activation feature map with the query weight matrix to construct the target query matrix H, and calculate the self-attention weight matrix ATT=softmax(HB T / (d) 0.5 ), d is the dimension size of the target fusion feature map, and the target value matrix A is multiplied by the self-attention weight matrix ATT to construct the third output feature map, which is the first enhanced feature map, the second enhanced feature map, the third enhanced feature map, and the fourth enhanced feature map corresponding to the selected fusion feature map.

3. The image-based auxiliary diagnosis method for rectal cancer according to claim 2, characterized in that: Training the colorectal cancer analysis model includes the following steps: Obtain several rectal cancer analysis training samples, which are microscopic images, and annotate the rectal cancer analysis training samples with rectal cancer analysis labels, and form a rectal cancer analysis training set with the annotated rectal cancer analysis training samples. Train the rectal cancer analysis model with the rectal cancer analysis training set to determine whether the training conditions are met. If the training conditions are met, output the trained rectal cancer analysis model; otherwise, train the rectal cancer analysis model with the rectal cancer analysis training set.

4. The image-based auxiliary diagnosis method for rectal cancer according to claim 3, characterized in that: The hyperparameters of the rectal cancer analysis model are simulated and set using a swarm optimization algorithm. The specific steps are as follows: Construct several hyperparameter simulation individuals; combine all hyperparameter simulation individuals into a population set and set the maximum number of iterations; Calculate the fitness corresponding to the hyperparameter simulation individual; Based on the fitness of individuals simulated by hyperparameters, the population set is iteratively updated through the population optimization algorithm; Until the number of iterations reaches the maximum number of iterations, the hyperparameters with the largest fitness are output to simulate the individual to set the hyperparameters for the rectal cancer analysis training set.

5. The image-based auxiliary diagnosis method for rectal cancer according to claim 4, characterized in that: The fitness corresponding to the hyperparameter simulation individual is calculated as follows: the hyperparameters of the colorectal cancer analysis model are set through the hyperparameter simulation individual, and then the colorectal cancer analysis model with the set hyperparameters is trained through the colorectal cancer analysis training set, and the accuracy of the colorectal cancer analysis model is used as the fitness corresponding to the hyperparameter simulation individual.

6. An image-based auxiliary diagnosis system for rectal cancer, characterized in that: The system uses an image-based auxiliary diagnosis method for rectal cancer as described in any one of claims 1 to 5, comprising: A microscopic image acquisition module is used to acquire microscopic images of patients; The rectal cancer analysis module is used to send the patient's microscopic examination images to the rectal cancer analysis model for processing and output the rectal cancer analysis label; The rectal cancer analysis model includes a branch feature extraction network, a feature alignment network, a feature collaboration network, a dynamic branch activation network, a feature competition network and a decision analysis network, wherein the branch feature extraction network includes a first residual block, a second residual block, a third residual block and a fourth residual block, and the branch feature extraction network is used to perform feature extraction operations on microscopic images, and the feature maps output by the first residual block, the second residual block, the third residual block and the fourth residual block are respectively recorded as the first feature map, the second feature map, the third feature map and the fourth feature map; the feature alignment network is used to extract the first feature map, the second feature map, the third feature map and the fourth feature map. The third feature map and the fourth feature map perform feature alignment operations to construct the corresponding first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map; the feature collaboration network is used to perform block operations on the first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map, respectively, and then construct feature fields for the first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map after the blocks, and generate a corresponding feature collaboration matrix, and the corresponding first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map are aligned based on the feature collaboration matrix. The third standard feature map and the fourth standard feature map are feature enhanced to construct the corresponding first fused feature map, the second fused feature map, the third fused feature map and the fourth fused feature map; the dynamic branch activation network is used to send the first feature map, the second feature map, the third feature map and the fourth feature map to the first multilayer perceptron for processing, and output the activation weights corresponding to the first feature map, the second feature map, the third feature map and the fourth feature map; the feature competition network is used to select the first fused feature map, the second fused feature map, the third fused feature map and the fourth fused feature map for self-attention operation based on the activation weights corresponding to the first feature map, the second feature map, the third feature map and the fourth feature map, and update the first fused feature map, the second fused feature map, the third fused feature map and the fourth fused feature map to construct the corresponding first enhanced feature map, the second enhanced feature map, the third enhanced feature map and the fourth enhanced feature map; the decision analysis network is used to perform weighted fusion operation on the first enhanced feature map, the second enhanced feature map, the third enhanced feature map and the fourth enhanced feature map based on the activation weights corresponding to the first feature map, the second feature map, the third feature map and the fourth feature map, and generate a rectal cancer analysis label.

Images