Image-based rectal cancer auxiliary diagnosis method and system

Abstract

The invention relates to the technical field of image processing, in particular to an image-based rectal cancer auxiliary diagnosis method and system. The invention provides an image-based rectal cancer auxiliary diagnosis system which comprises a microscopic examination image acquisition module and a rectal cancer analysis module. A microscopic examination picture of a patient is analyzed through a rectal cancer analysis model, and in the rectal cancer analysis model, feature fields of different blocks in a first standard feature map, a second standard feature map, a third standard feature map and a fourth standard feature map are interacted through a feature collaborative network to reinforce the feature maps; the method can improve the feature expression ability and global information expression ability of the feature maps, and further reinforces the first fusion feature map, the second fusion feature map, the third fusion feature map and the fourth fusion feature map through the concentration degrees of different branches, thereby enhancing the universality of a rectal cancer analysis model. And the analysis accuracy of the rectal cancer analysis model is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an image-based auxiliary diagnosis method and system for rectal cancer. Background Art

[0002] In the field of medical imaging diagnosis, early detection and diagnosis of rectal cancer are of great significance for improving the survival rate of patients. However, traditional methods of rectal cancer diagnosis mainly rely on doctors' manual observation and analysis of medical images (such as endoscopic images, pathological slice images, etc.). Although this manual diagnosis method can provide reliable diagnostic results to a certain extent, it also has significant limitations. First, doctors need to process a large amount of medical imaging data for a long time and with high intensity, especially in the screening stage, which greatly increases the workload of doctors. With the continuous growth of medical needs and the relative shortage of high-quality medical resources, the workload of doctors is increasing, which is easy to cause fatigue and decreased attention, which may affect the accuracy of diagnosis. Summary of the invention

[0003] The present invention analyzes the microscopic images of patients through a rectal cancer analysis model to assist in the diagnosis of rectal cancer, which can reduce the workload of doctors. In the rectal cancer analysis model, the feature fields of different blocks in the first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map are interacted through a feature collaborative network to strengthen the feature map, which can improve the feature expression ability and global information expression ability of the feature map. The first fusion feature map, the second fusion feature map, the third fusion feature map and the fourth fusion feature map are further strengthened through the degree of concentration of different branches, thereby enhancing the versatility of the rectal cancer analysis model and improving the analysis accuracy of the rectal cancer analysis model.

[0004] The invention provides an image-based auxiliary diagnosis method for rectal cancer, comprising: Obtain the patient's microscopic images, and then send the patient's microscopic images to the rectal cancer analysis model for processing, and output the rectal cancer analysis label; The rectal cancer analysis model includes a branch feature extraction network, a feature alignment network, a feature collaboration network, a dynamic branch activation network, a feature competition network and a decision analysis network, wherein the branch feature extraction network includes a first residual block, a second residual block, a third residual block and a fourth residual block, and the branch feature extraction network is used to perform feature extraction operations on microscopic images, and the feature maps output by the first residual block, the second residual block, the third residual block and the fourth residual block are respectively recorded as the first feature map, the second feature map, the third feature map and the fourth feature map; the feature alignment network is used to perform feature alignment on the first feature map, the second feature map, the third feature map and the fourth feature map The first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map are respectively operated to construct the corresponding first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map; the feature synergy network is used to perform block operations on the first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map, respectively, and then construct feature fields for the first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map after the block, and generate a corresponding feature synergy matrix, and based on the feature synergy matrix, the corresponding first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map are feature enhanced to construct the corresponding first fusion feature map, the second fused feature map, the third fused feature map and the fourth fused feature map; the dynamic branch activation network is used to send the first feature map, the second feature map, the third feature map and the fourth feature map to the first multilayer perceptron for processing, and output the activation weights corresponding to the first feature map, the second feature map, the third feature map and the fourth feature map; the feature competition network is used to select the first fused feature map, the second fused feature map, the third fused feature map and the fourth fused feature map for self-attention operation based on the activation weights corresponding to the first feature map, the second feature map, the third feature map and the fourth feature map, and perform self-attention operation on the first fused feature map, the second fused feature map, the third feature map and the fourth feature map The three fused feature maps and the fourth fused feature map are updated to construct the corresponding first enhanced feature map, the second enhanced feature map, the third enhanced feature map and the fourth enhanced feature map; the decision analysis network is used to perform a weighted fusion operation on the first enhanced feature map, the second enhanced feature map, the third enhanced feature map and the fourth enhanced feature map based on the activation weights corresponding to the first feature map, the second feature map, the third enhanced feature map and the fourth feature map, that is, the first enhanced feature map, the second enhanced feature map, the third enhanced feature map and the fourth enhanced feature map are multiplied by the corresponding activation weights, and then spliced ​​according to the channels, and then convolution and full connection operations are performed to generate a rectal cancer analysis label.

[0005] Preferably, the feature alignment operation is performed on the first feature map, the second feature map, the third feature map and the fourth feature map through a feature alignment network to construct corresponding first standard feature map, second standard feature map, third standard feature map and fourth standard feature map, which specifically includes the following steps: The first feature map, the second feature map, the third feature map and the fourth feature map are traversed, and the selected feature map is recorded as the target feature map, and then the target feature map is expanded in one dimension to construct a target feature vector, and then a position encoding operation is performed on the target feature map to construct a position vector, and the target feature vector is added to the position vector to construct an intermediate feature vector, and the intermediate feature vector is sent to the second multi-layer perceptron for processing to obtain an output feature vector, and then the output feature vector is folded into the first output feature map, and the size of the first output feature map is consistent with the microscopic image; the first output feature map is the first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map corresponding to the selected feature map.

[0006] Preferably, the first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map are respectively divided into blocks through a feature collaboration network, and then the feature fields are respectively constructed for the divided first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map, and a corresponding feature collaboration matrix is ​​generated. Based on the feature collaboration matrix, the corresponding first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map are feature enhanced to construct the corresponding first fused feature map, the second fused feature map, the third fused feature map and the fourth fused feature map, which specifically includes the following steps: Traverse the first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map, record the selected standard feature map as the target standard feature map, perform a block operation on the target standard feature map through a sliding window with a step size of 1, construct several local feature maps, and record the central coordinate position of the i-th local feature map as the feature block position W i , i=1, 2, 3, ..., I, I is the total number of local feature maps, and the maximum value in the i-th local feature map is selected as the feature block strength Q i , the gradient direction calculated by the Sobel operator in the i-th local feature map is recorded as the feature block direction F i , the feature block positions W corresponding to all local feature maps i , feature block strength Q i and feature block direction F i Form a feature field, and for each local feature map, calculate the feature coordination weight corresponding to the i-th local feature map , where P i (t) is the number of i The feature coordination weights corresponding to the local feature maps, k=1, 2, 3, ..., I, and k≠ i , that is, select all local feature maps except the i-th local feature map, μ is the feature field strength coefficient, |W i , W k| is the distance between the ith local feature map and the kth local feature map, and cos() is the cosine calculation; until the number of iterations reaches the maximum number of iterations, the feature synergy weights corresponding to all local feature maps are composed of a feature synergy matrix according to the positions of the local feature maps, and the target standard feature map is multiplied by the corresponding feature synergy matrix to construct the second output feature map corresponding to the target standard feature map. The second output feature map is the first fused feature map, the second fused feature map, the third fused feature map, and the fourth fused feature map corresponding to the selected standard feature map.

[0007] Preferably, a feature competition network is used to select the first fused feature map, the second fused feature map, the third fused feature map and the fourth fused feature map for self-attention operation based on the activation weights corresponding to the first feature map, the second feature map, the third feature map and the fourth feature map, and the first fused feature map, the second fused feature map, the third fused feature map and the fourth fused feature map are updated to construct the corresponding first enhanced feature map, the second enhanced feature map, the third enhanced feature map and the fourth enhanced feature map, which specifically includes the following steps: Select the first feature map, the second feature map, the third feature map or the fourth feature map corresponding to the maximum activation weight as the target activation feature map; Traverse the first fusion feature map, the second fusion feature map, the third fusion feature map and the fourth fusion feature map, record the selected fusion feature map as the target fusion feature map, multiply the target fusion feature map with the value weight matrix and the key weight matrix to construct the target value matrix A and the target key matrix B, multiply the target activation feature map with the query weight matrix to construct the target query matrix H, and calculate the self-attention weight matrix ATT=softmax(HB T / (d) 0.5 ), d is the dimension size of the target fusion feature map, and the target value matrix A is multiplied by the self-attention weight matrix ATT to construct the third output feature map, which is the first enhanced feature map, the second enhanced feature map, the third enhanced feature map, and the fourth enhanced feature map corresponding to the selected fusion feature map.

[0008] Preferably, training the rectal cancer analysis model specifically includes the following steps: Obtain several rectal cancer analysis training samples, which are microscopic images, and annotate the rectal cancer analysis training samples with rectal cancer analysis labels, and form a rectal cancer analysis training set with the annotated rectal cancer analysis training samples. Train the rectal cancer analysis model with the rectal cancer analysis training set to determine whether the training conditions are met. If the training conditions are met, output the trained rectal cancer analysis model; otherwise, train the rectal cancer analysis model with the rectal cancer analysis training set.

[0009] Preferably, the hyperparameters of the rectal cancer analysis model are simulated and set by a population optimization algorithm, and the specific steps are as follows: Construct several hyperparameter simulation individuals; combine all hyperparameter simulation individuals into a population set and set the maximum number of iterations; Calculate the fitness corresponding to the hyperparameter simulation individual; Based on the fitness of individuals simulated by hyperparameters, the population set is iteratively updated through the population optimization algorithm; Until the number of iterations reaches the maximum number of iterations, the hyperparameters with the largest fitness are output to simulate the individual to set the hyperparameters for the rectal cancer analysis training set.

[0010] Preferably, the fitness corresponding to the hyperparameter simulation individual is calculated as follows: the hyperparameters of the rectal cancer analysis model are set through the hyperparameter simulation individual, and then the rectal cancer analysis model with the set hyperparameters is trained through the rectal cancer analysis training set, and the accuracy of the rectal cancer analysis model is used as the fitness corresponding to the hyperparameter simulation individual.

[0011] The present invention also provides an image-based rectal cancer auxiliary diagnosis system, comprising: A microscopic image acquisition module is used to acquire microscopic images of patients; The rectal cancer analysis module is used to send the patient's microscopic examination images to the rectal cancer analysis model for processing and output the rectal cancer analysis label; The rectal cancer analysis model includes a branch feature extraction network, a feature alignment network, a feature collaboration network, a dynamic branch activation network, a feature competition network and a decision analysis network, wherein the branch feature extraction network includes a first residual block, a second residual block, a third residual block and a fourth residual block, and the branch feature extraction network is used to perform feature extraction operations on microscopic images, and the feature maps output by the first residual block, the second residual block, the third residual block and the fourth residual block are respectively recorded as the first feature map, the second feature map, the third feature map and the fourth feature map; the feature alignment network is used to extract the first feature map, the second feature map, the third feature map and the fourth feature map. The third feature map and the fourth feature map perform feature alignment operations to construct the corresponding first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map; the feature collaboration network is used to perform block operations on the first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map, respectively, and then construct feature fields for the first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map after the blocks, and generate a corresponding feature collaboration matrix, and the corresponding first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map are aligned based on the feature collaboration matrix. The third standard feature map and the fourth standard feature map are feature enhanced to construct the corresponding first fused feature map, the second fused feature map, the third fused feature map and the fourth fused feature map; the dynamic branch activation network is used to send the first feature map, the second feature map, the third feature map and the fourth feature map to the first multilayer perceptron for processing, and output the activation weights corresponding to the first feature map, the second feature map, the third feature map and the fourth feature map; the feature competition network is used to select the first fused feature map, the second fused feature map, the third fused feature map and the fourth fused feature map for self-attention operation based on the activation weights corresponding to the first feature map, the second feature map, the third feature map and the fourth feature map, and update the first fused feature map, the second fused feature map, the third fused feature map and the fourth fused feature map to construct the corresponding first enhanced feature map, the second enhanced feature map, the third enhanced feature map and the fourth enhanced feature map; the decision analysis network is used to perform weighted fusion operation on the first enhanced feature map, the second enhanced feature map, the third enhanced feature map and the fourth enhanced feature map based on the activation weights corresponding to the first feature map, the second feature map, the third feature map and the fourth feature map, and generate a rectal cancer analysis label.

[0012] The present invention has the following advantages: The present invention analyzes the microscopic images of patients through a rectal cancer analysis model to assist in the diagnosis of rectal cancer, which can reduce the workload of doctors. In the rectal cancer analysis model, the feature fields of different blocks in the first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map are interacted through a feature collaborative network to strengthen the feature map, which can improve the feature expression ability and global information expression ability of the feature map. The first fusion feature map, the second fusion feature map, the third fusion feature map and the fourth fusion feature map are further strengthened through the degree of concentration of different branches, thereby enhancing the versatility of the rectal cancer analysis model and improving the analysis accuracy of the rectal cancer analysis model. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is a schematic diagram of the structure of the rectal cancer analysis model used in an embodiment of the present invention.

[0014] Figure 2 It is a schematic diagram of the structure of the image-based rectal cancer auxiliary diagnosis system adopted in an embodiment of the present invention. DETAILED DESCRIPTION

[0015] In order to enable persons skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0016] Embodiment 1, an image-based auxiliary diagnosis method for rectal cancer, comprising: Obtain the patient's microscopic images. Here, microscopic images refer to images taken of the patient's rectal mucosal surface by narrow-band imaging endoscopes. The features of the rectal blood vessels and mucosal surface in the microscopic images can assist in the diagnosis of rectal cancer. Then, the patient's microscopic images are sent to the rectal cancer analysis model for processing, and rectal cancer analysis labels are output. Rectal cancer analysis labels generally include benign, malignant, and normal, and malignant means that the patient has rectal cancer, while benign generally refers to rectal lesions such as rectal polyps or rectal adenomas. Using the rectal cancer analysis model to assist in the diagnosis of rectal cancer can reduce the workload of doctors. See also Figure 1The rectal cancer analysis model includes a branch feature extraction network, a feature alignment network, a feature collaboration network, a dynamic branch activation network, a feature competition network and a decision analysis network, wherein the branch feature extraction network is established based on the ResNet50 model, including the first residual block, the second residual block, the third residual block and the fourth residual block, and the branch feature extraction network is used to perform feature extraction operations on the microscopic images, and the feature maps output by the first residual block, the second residual block, the third residual block and the fourth residual block are respectively recorded as the first feature map, the second feature map, the third feature map and the fourth feature map. It needs to be explained The first residual block, the second residual block, the third residual block and the fourth residual block are regarded as feature extraction branches at different depths, the first feature map corresponds to low-depth features, such as edges and color gradients, the second feature map corresponds to lower-depth features, such as texture features, the third feature map corresponds to medium-depth features, such as local structural features, and the fourth feature map corresponds to high-depth features, such as global semantic features, and the first residual block, the second residual block, the third residual block and the fourth residual block are set with reference to the ResNet50 model, and all include multi-level convolution operations; the first feature map corresponds to low-depth features, such as edges and color gradients, the second feature map corresponds to lower-depth features, such as texture features, the third feature map corresponds to medium-depth features, such as local structural features, and the fourth feature map corresponds to high-depth features, such as global semantic features. The first residual block, the second residual block, the third residual block and the fourth residual block are set with reference to the ResNet50 model, and all include multi-level convolution operations; the The feature alignment network is used to perform feature alignment operations on the first feature map, the second feature map, the third feature map and the fourth feature map to construct corresponding first standard feature maps, second standard feature maps, third standard feature maps and fourth standard feature maps; the feature synergy network is used to perform block operations on the first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map, respectively, and then construct feature fields for the first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map after block, and generate corresponding feature synergy matrices, and based on the feature synergy matrix, feature enhancement is performed on the corresponding first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map to construct corresponding first fusion feature maps, second fusion feature maps, third fusion feature maps and fourth fusion feature maps. It should be noted that the feature fields of the first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map after block are constructed respectively, so that the interaction strength and direction between different blocks in the feature map can be learned, and feature fusion is performed through the interaction of feature fields between different blocks, so that higher quality feature maps can be achieved;The dynamic branch activation network is used to send the first feature map, the second feature map, the third feature map and the fourth feature map to the first multilayer perceptron for processing, and output the activation weights corresponding to the first feature map, the second feature map, the third feature map and the fourth feature map. The activation weights represent the focus on different branch features in the microscopic image. For example, for small lesion images, the branches focus on tiny features, and for large lesion images, the branches focus on global context features. In addition, there may be noise, artifacts, and uneven contrast in the microscopic image. By activating different branch features, the robustness of the rectal cancer analysis model in the face of these problems can be enhanced. For example, if the feature extracted by a branch is interfered by noise, the dynamic branch mechanism can weaken the role of the branch and rely on other branches instead. The feature competition network is used to select the first fused feature map, the second fused feature map, the third fused feature map and the fourth fused feature map for self-attention operation based on the activation weights corresponding to the first feature map, the second feature map, the third feature map and the fourth feature map, and update the first fused feature map, the second fused feature map, the third fused feature map and the fourth fused feature map to construct the corresponding first enhanced feature map, the second enhanced feature map, the third enhanced feature map and the fourth enhanced feature map; the decision analysis network is used to perform a weighted fusion operation on the first enhanced feature map, the second enhanced feature map, the third enhanced feature map and the fourth enhanced feature map based on the activation weights corresponding to the first feature map, the second feature map, the third feature map and the fourth feature map, and generate a rectal cancer analysis label; The present application analyzes the microscopic images of patients through a rectal cancer analysis model to assist in the diagnosis of rectal cancer, which can reduce the workload of doctors. In the rectal cancer analysis model, the feature fields of different blocks in the first standard feature map, the second standard feature map, the third standard feature map, and the fourth standard feature map are interacted through a feature collaborative network to strengthen the feature map, which can improve the feature expression ability and global information expression ability of the feature map. The first fusion feature map, the second fusion feature map, the third fusion feature map, and the fourth fusion feature map are further strengthened through the degree of focus of different branches, thereby enhancing the versatility of the rectal cancer analysis model and improving the analysis accuracy of the rectal cancer analysis model.

[0017] The first feature map, the second feature map, the third feature map and the fourth feature map are subjected to a feature alignment operation through a feature alignment network to construct corresponding first standard feature maps, second standard feature maps, third standard feature maps and fourth standard feature maps, specifically comprising the following steps: Traverse the first feature map, the second feature map, the third feature map and the fourth feature map, and record the selected feature map as the target feature map, then expand the target feature map in one dimension, construct the target feature vector, and then perform position encoding operation on the target feature map to construct the position vector. The position encoding operation refers to the position encoding of the Transformer model, and uses sine and cosine functions. The spatial structure in the feature map can be retained through the position encoding operation. The target feature vector and the position vector are added to construct an intermediate feature vector, and the intermediate feature vector is sent to the second multi-layer perceptron for processing to obtain the output feature vector, and then the output feature vector is folded into the first output feature map. The folding here refers to restoring the output feature vector to a three-dimensional data structure, and the size of the first output feature map is consistent with the microscopic image; the first output feature map is the first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map corresponding to the selected feature map.

[0018] The first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map are respectively divided into blocks through a feature collaboration network, and then the feature fields are respectively constructed for the divided first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map, and a corresponding feature collaboration matrix is ​​generated. Based on the feature collaboration matrix, the corresponding first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map are feature enhanced to construct the corresponding first fusion feature map, the second fusion feature map, the third fusion feature map and the fourth fusion feature map, which specifically includes the following steps: Traverse the first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map, record the selected standard feature map as the target standard feature map, divide the target standard feature map into blocks through a sliding window with a step size of 1, construct several local feature maps, and the size of the sliding window is generally 3×3, record the central coordinate position of the i-th local feature map as the feature block position W i , i=1, 2, 3, ..., I, I is the total number of local feature maps, and the maximum value in the i-th local feature map is selected as the feature block strength Q i , the gradient direction calculated by the Sobel operator in the i-th local feature map is recorded as the feature block direction F i , the feature block positions W corresponding to all local feature maps i , feature block strength Q i and feature block direction F i Form a feature field, and for each local feature map, calculate the feature coordination weight corresponding to the i-th local feature map , where P i (t) is the number of iThe feature coordination weights corresponding to the local feature maps, k=1, 2, 3, ..., I, and k≠ i , that is, select all local feature maps except the i-th local feature map, μ is the feature field strength coefficient, which is set manually, |W i , W k | is the distance between the ith local feature map and the kth local feature map, and cos() is the cosine calculation; until the number of iterations reaches the maximum number of iterations, the feature synergy weights corresponding to all local feature maps are composed of a feature synergy matrix according to the positions of the local feature maps, and the target standard feature map is multiplied by the corresponding feature synergy matrix to construct the second output feature map corresponding to the target standard feature map. The second output feature map is the first fused feature map, the second fused feature map, the third fused feature map, and the fourth fused feature map corresponding to the selected standard feature map.

[0019] The feature competition network is used to select the first fused feature map, the second fused feature map, the third fused feature map and the fourth fused feature map for self-attention operation based on the activation weights corresponding to the first feature map, the second feature map, the third feature map and the fourth feature map, and the first fused feature map, the second fused feature map, the third fused feature map and the fourth fused feature map are updated to construct the corresponding first enhanced feature map, the second enhanced feature map, the third enhanced feature map and the fourth enhanced feature map, specifically including the following steps: Select the first feature map, the second feature map, the third feature map or the fourth feature map corresponding to the maximum activation weight as the target activation feature map; Traverse the first fusion feature map, the second fusion feature map, the third fusion feature map and the fourth fusion feature map, record the selected fusion feature map as the target fusion feature map, multiply the target fusion feature map with the value weight matrix and the key weight matrix to construct the target value matrix A and the target key matrix B, multiply the target activation feature map with the query weight matrix to construct the target query matrix H, and calculate the self-attention weight matrix ATT=softmax(HB T / (d) 0.5 ), d is the dimension size of the target fusion feature map, and multiplies the target value matrix A and the self-attention weight matrix ATT to construct the third output feature map. The third output feature map is the first enhanced feature map, the second enhanced feature map, the third enhanced feature map and the fourth enhanced feature map corresponding to the selected fusion feature map; the value weight matrix, key weight matrix and query weight matrix here refer to the self-attention mechanism setting in the Transformer model.

[0020] Training the colorectal cancer analysis model includes the following steps: Obtain several rectal cancer analysis training samples, which are microscopic images actually collected by operators, and annotate the rectal cancer analysis training samples with rectal cancer analysis labels, and form a rectal cancer analysis training set with the annotated rectal cancer analysis training samples. Train the rectal cancer analysis model with the rectal cancer analysis training set to determine whether the training conditions are met. The training conditions are generally that the accuracy of the rectal cancer analysis model meets expectations. If the training conditions are met, output the trained rectal cancer analysis model; otherwise, train the rectal cancer analysis model with the rectal cancer analysis training set.

[0021] In order to further improve the accuracy of the rectal cancer analysis model, the hyperparameters of the rectal cancer analysis model are simulated and set through a swarm optimization algorithm. The specific steps are as follows: Constructing several hyperparameter simulation individuals, including hyperparameters in the colorectal cancer analysis model, such as the maximum number of training iterations and the learning rate, etc. The hyperparameter simulation individuals are constructed by selecting random values ​​within the value range of the hyperparameters to assign values ​​to the hyperparameters; forming all the hyperparameter simulation individuals into a population set, and setting the maximum number of iterations; Calculate the fitness corresponding to the hyperparameter simulation individual. The calculation method is as follows: set the hyperparameters of the rectal cancer analysis model through the hyperparameter simulation individual, then train the rectal cancer analysis model with the set hyperparameters through the rectal cancer analysis training set, and use the accuracy of the rectal cancer analysis model as the fitness corresponding to the hyperparameter simulation individual; Based on the fitness of individuals simulated by hyperparameters, the population set is iteratively updated through the sparrow search algorithm; Until the number of iterations reaches the maximum number of iterations, the hyperparameters with the largest fitness are output to simulate the individual to set the hyperparameters for the rectal cancer analysis training set.

[0022] Example 2, an image-based auxiliary diagnosis system for rectal cancer, such as Figure 2 As shown, including: The microscopic image acquisition module is used to obtain the microscopic images of the patient. The microscopic images here refer to the images taken of the patient's rectal mucosal surface by a narrow-band imaging endoscope. The features of the rectal blood vessels and mucosal surface in the microscopic images can assist in the diagnosis of rectal cancer. The rectal cancer analysis module is used to send the patient's microscopic examination images to the rectal cancer analysis model for processing and output the rectal cancer analysis labels. The rectal cancer analysis labels generally include benign, malignant and normal, and malignant refers to rectal cancer, while benign generally refers to rectal lesions such as rectal polyps or rectal adenomas. The rectal cancer analysis model can assist in the diagnosis of rectal cancer and reduce the workload of doctors. The rectal cancer analysis model includes a branch feature extraction network, a feature alignment network, a feature collaboration network, a dynamic branch activation network, a feature competition network and a decision analysis network, wherein the branch feature extraction network is established based on the ResNet50 model, including the first residual block, the second residual block, the third residual block and the fourth residual block, and the branch feature extraction network is used to perform feature extraction operations on the microscopic images, and the feature maps output by the first residual block, the second residual block, the third residual block and the fourth residual block are respectively recorded as the first feature map, the second feature map, the third feature map and the fourth feature map. It needs to be explained The first residual block, the second residual block, the third residual block and the fourth residual block are regarded as feature extraction branches at different depths, the first feature map corresponds to low-depth features, such as edges and color gradients, the second feature map corresponds to lower-depth features, such as texture features, the third feature map corresponds to medium-depth features, such as local structural features, and the fourth feature map corresponds to high-depth features, such as global semantic features, and the first residual block, the second residual block, the third residual block and the fourth residual block are set with reference to the ResNet50 model, and all include multi-level convolution operations; the first feature map corresponds to low-depth features, such as edges and color gradients, the second feature map corresponds to lower-depth features, such as texture features, the third feature map corresponds to medium-depth features, such as local structural features, and the fourth feature map corresponds to high-depth features, such as global semantic features. The first residual block, the second residual block, the third residual block and the fourth residual block are set with reference to the ResNet50 model, and all include multi-level convolution operations; the The feature alignment network is used to perform feature alignment operations on the first feature map, the second feature map, the third feature map and the fourth feature map to construct corresponding first standard feature maps, second standard feature maps, third standard feature maps and fourth standard feature maps; the feature synergy network is used to perform block operations on the first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map, respectively, and then construct feature fields for the first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map after block, and generate corresponding feature synergy matrices, and based on the feature synergy matrix, feature enhancement is performed on the corresponding first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map to construct corresponding first fusion feature maps, second fusion feature maps, third fusion feature maps and fourth fusion feature maps. It should be noted that the feature fields of the first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map after block are constructed respectively, so that the interaction strength and direction between different blocks in the feature map can be learned, and feature fusion is performed through the interaction of feature fields between different blocks, so that higher quality feature maps can be achieved;The dynamic branch activation network is used to send the first feature map, the second feature map, the third feature map and the fourth feature map to the first multilayer perceptron for processing, and output the activation weights corresponding to the first feature map, the second feature map, the third feature map and the fourth feature map. The activation weights represent the focus on different branch features in the microscopic image. For example, for small lesion images, the branches focus on tiny features, and for large lesion images, the branches focus on global context features. In addition, there may be noise, artifacts, and uneven contrast in the microscopic image. By activating different branch features, the robustness of the rectal cancer analysis model in the face of these problems can be enhanced. For example, if the feature extracted by a branch is interfered by noise, the dynamic branch mechanism can weaken the role of the branch and rely on other branches instead. The feature competition network is used to select the first fusion feature map, the second fusion feature map, the third fusion feature map and the fourth fusion feature map for self-attention operation based on the activation weights corresponding to the first feature map, the second feature map, the third fusion feature map and the fourth feature map, and update the first fusion feature map, the second fusion feature map, the third fusion feature map and the fourth fusion feature map to construct the corresponding first enhanced feature map, the second enhanced feature map, the third enhanced feature map and the fourth enhanced feature map; the decision analysis network is used to perform weighted fusion operation on the first enhanced feature map, the second enhanced feature map, the third enhanced feature map and the fourth enhanced feature map based on the activation weights corresponding to the first feature map, the second feature map, the third feature map and the fourth feature map, and generate a rectal cancer analysis label. ;

[0023] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all such improvements and changes should fall within the scope of protection of the appended claims of the present invention. Parts not described in detail in this specification belong to the prior art known to those skilled in the art.

Claims

1. An image-based auxiliary diagnosis method for rectal cancer, characterized in that: include: Obtain the patient's microscopic images, and then send the patient's microscopic images to the rectal cancer analysis model for processing, and output the rectal cancer analysis label; The rectal cancer analysis model includes a branch feature extraction network, a feature alignment network, a feature collaboration network, a dynamic branch activation network, a feature competition network and a decision analysis network, wherein the branch feature extraction network includes a first residual block, a second residual block, a third residual block and a fourth residual block, and the branch feature extraction network is used to perform feature extraction operations on microscopic images, and the feature maps output by the first residual block, the second residual block, the third residual block and the fourth residual block are respectively recorded as the first feature map, the second feature map, the third feature map and the fourth feature map; the feature alignment network is used to perform feature alignment operations on the first feature map, the second feature map, the third feature map and the fourth feature map to construct corresponding first standard feature maps, second standard feature maps, third standard feature maps and fourth standard feature maps; The feature synergy network is used to perform block operations on the first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map, respectively, and then construct feature fields for the first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map after blocking, and generate corresponding feature synergy matrices. Based on the feature synergy matrix, the corresponding first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map are feature enhanced to construct the corresponding first fusion feature map, the second fusion feature map, the third fusion feature map and the fourth fusion feature map; the dynamic branch activation network is used to send the first feature map, the second feature map, the third feature map and the fourth feature map to the first multilayer perceptron for processing, and output the first feature map, the second feature map, the third feature map and the fourth feature map. The activation weights corresponding to the feature map and the fourth feature map; the feature competition network is used to select the first fused feature map, the second fused feature map, the third fused feature map and the fourth fused feature map for self-attention operation based on the activation weights corresponding to the first feature map, the second feature map, the third feature map and the fourth feature map, and update the first fused feature map, the second fused feature map, the third fused feature map and the fourth fused feature map to construct the corresponding first enhanced feature map, the second enhanced feature map, the third enhanced feature map and the fourth enhanced feature map; the decision analysis network is used to perform weighted fusion operation on the first enhanced feature map, the second enhanced feature map, the third enhanced feature map and the fourth enhanced feature map based on the activation weights corresponding to the first feature map, the second feature map, the third feature map and the fourth feature map, and generate a rectal cancer analysis label.

2. The image-based auxiliary diagnosis method for rectal cancer according to claim 1, characterized in that: The first feature map, the second feature map, the third feature map and the fourth feature map are subjected to a feature alignment operation through a feature alignment network to construct corresponding first standard feature maps, second standard feature maps, third standard feature maps and fourth standard feature maps, specifically comprising the following steps: The first feature map, the second feature map, the third feature map and the fourth feature map are traversed, and the selected feature map is recorded as the target feature map, and then the target feature map is expanded in one dimension to construct a target feature vector, and then a position encoding operation is performed on the target feature map to construct a position vector, and the target feature vector is added to the position vector to construct an intermediate feature vector, and the intermediate feature vector is sent to the second multi-layer perceptron for processing to obtain an output feature vector, and then the output feature vector is folded into the first output feature map, and the size of the first output feature map is consistent with the microscopic image; the first output feature map is the first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map corresponding to the selected feature map.

3. The image-based auxiliary diagnosis method for rectal cancer according to claim 2, characterized in that: The first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map are respectively divided into blocks through a feature collaboration network, and then the feature fields are respectively constructed for the divided first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map, and a corresponding feature collaboration matrix is ​​generated. Based on the feature collaboration matrix, the corresponding first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map are feature enhanced to construct the corresponding first fusion feature map, the second fusion feature map, the third fusion feature map and the fourth fusion feature map, which specifically includes the following steps: Traverse the first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map, record the selected standard feature map as the target standard feature map, perform a block operation on the target standard feature map through a sliding window with a step size of 1, construct several local feature maps, and record the central coordinate position of the i-th local feature map as the feature block position W i , i=1, 2, 3, ..., I, I is the total number of local feature maps, and the maximum value in the i-th local feature map is recorded as the feature block strength Q i , the gradient direction calculated by the Sobel operator in the i-th local feature map is recorded as the feature block direction F i , the feature block positions W corresponding to all local feature maps i , feature block strength Q i and feature block direction F i Form a feature field, and for each local feature map, calculate the feature coordination weight corresponding to the i-th local feature map , where P i (t) is the number of i The feature coordination weights corresponding to the local feature maps, k=1, 2, 3, ..., I, and k≠ i , that is, select all local feature maps except the i-th local feature map, μ is the feature field strength coefficient, |W i , W k | is the distance between the ith local feature map and the kth local feature map, and cos() is the cosine calculation; until the number of iterations reaches the maximum number of iterations, the feature synergy weights corresponding to all local feature maps are composed of a feature synergy matrix according to the positions of the local feature maps, and the target standard feature map is multiplied by the corresponding feature synergy matrix to construct the second output feature map corresponding to the target standard feature map. The second output feature map is the first fused feature map, the second fused feature map, the third fused feature map, and the fourth fused feature map corresponding to the selected standard feature map.

4. The image-based auxiliary diagnosis method for rectal cancer according to claim 3, characterized in that: The feature competition network is used to select the first fused feature map, the second fused feature map, the third fused feature map and the fourth fused feature map for self-attention operation based on the activation weights corresponding to the first feature map, the second feature map, the third feature map and the fourth feature map, and the first fused feature map, the second fused feature map, the third fused feature map and the fourth fused feature map are updated to construct the corresponding first enhanced feature map, the second enhanced feature map, the third enhanced feature map and the fourth enhanced feature map, specifically including the following steps: Select the first feature map, the second feature map, the third feature map or the fourth feature map corresponding to the maximum activation weight as the target activation feature map; Traverse the first fusion feature map, the second fusion feature map, the third fusion feature map and the fourth fusion feature map, record the selected fusion feature map as the target fusion feature map, multiply the target fusion feature map with the value weight matrix and the key weight matrix to construct the target value matrix A and the target key matrix B, multiply the target activation feature map with the query weight matrix to construct the target query matrix H, and calculate the self-attention weight matrix ATT=softmax(HB T / (d) 0.5 ), d is the dimension size of the target fusion feature map, and the target value matrix A is multiplied by the self-attention weight matrix ATT to construct the third output feature map, which is the first enhanced feature map, the second enhanced feature map, the third enhanced feature map, and the fourth enhanced feature map corresponding to the selected fusion feature map.

5. The image-based auxiliary diagnosis method for rectal cancer according to claim 4, characterized in that: Training the colorectal cancer analysis model includes the following steps: Obtain several rectal cancer analysis training samples, which are microscopic images, and annotate the rectal cancer analysis training samples with rectal cancer analysis labels, and form a rectal cancer analysis training set with the annotated rectal cancer analysis training samples. Train the rectal cancer analysis model with the rectal cancer analysis training set to determine whether the training conditions are met. If the training conditions are met, output the trained rectal cancer analysis model; otherwise, train the rectal cancer analysis model with the rectal cancer analysis training set.

6. The image-based auxiliary diagnosis method for rectal cancer according to claim 5, characterized in that: The hyperparameters of the rectal cancer analysis model are simulated and set using a swarm optimization algorithm. The specific steps are as follows: Construct several hyperparameter simulation individuals; combine all hyperparameter simulation individuals into a population set and set the maximum number of iterations; Calculate the fitness corresponding to the hyperparameter simulation individual; Based on the fitness of individuals simulated by hyperparameters, the population set is iteratively updated through the population optimization algorithm; Until the number of iterations reaches the maximum number of iterations, the hyperparameters with the largest fitness are output to simulate the individual to set the hyperparameters for the rectal cancer analysis training set.

7. The image-based auxiliary diagnosis method for rectal cancer according to claim 6, characterized in that: The fitness corresponding to the hyperparameter simulation individual is calculated as follows: the hyperparameters of the colorectal cancer analysis model are set through the hyperparameter simulation individual, and then the colorectal cancer analysis model with the set hyperparameters is trained through the colorectal cancer analysis training set, and the accuracy of the colorectal cancer analysis model is used as the fitness corresponding to the hyperparameter simulation individual.

8. An image-based auxiliary diagnosis system for rectal cancer, characterized in that: The system uses an image-based auxiliary diagnosis method for rectal cancer as described in any one of claims 1 to 7, comprising: A microscopic image acquisition module is used to acquire microscopic images of patients; The rectal cancer analysis module is used to send the patient's microscopic examination images to the rectal cancer analysis model for processing and output the rectal cancer analysis label; The rectal cancer analysis model includes a branch feature extraction network, a feature alignment network, a feature collaboration network, a dynamic branch activation network, a feature competition network and a decision analysis network, wherein the branch feature extraction network includes a first residual block, a second residual block, a third residual block and a fourth residual block, and the branch feature extraction network is used to perform feature extraction operations on microscopic images, and the feature maps output by the first residual block, the second residual block, the third residual block and the fourth residual block are respectively recorded as the first feature map, the second feature map, the third feature map and the fourth feature map; the feature alignment network is used to extract the first feature map, the second feature map, the third feature map and the fourth feature map. The third feature map and the fourth feature map perform feature alignment operations to construct the corresponding first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map; the feature collaboration network is used to perform block operations on the first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map, respectively, and then construct feature fields for the first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map after the blocks, and generate a corresponding feature collaboration matrix, and the corresponding first standard feature map, the second standard feature map, the third standard feature map and the fourth standard feature map are aligned based on the feature collaboration matrix. The third standard feature map and the fourth standard feature map are feature enhanced to construct the corresponding first fused feature map, the second fused feature map, the third fused feature map and the fourth fused feature map; the dynamic branch activation network is used to send the first feature map, the second feature map, the third feature map and the fourth feature map to the first multilayer perceptron for processing, and output the activation weights corresponding to the first feature map, the second feature map, the third feature map and the fourth feature map; the feature competition network is used to select the first fused feature map, the second fused feature map, the third fused feature map and the fourth fused feature map for self-attention operation based on the activation weights corresponding to the first feature map, the second feature map, the third feature map and the fourth feature map, and update the first fused feature map, the second fused feature map, the third fused feature map and the fourth fused feature map to construct the corresponding first enhanced feature map, the second enhanced feature map, the third enhanced feature map and the fourth enhanced feature map; the decision analysis network is used to perform weighted fusion operation on the first enhanced feature map, the second enhanced feature map, the third enhanced feature map and the fourth enhanced feature map based on the activation weights corresponding to the first feature map, the second feature map, the third feature map and the fourth feature map, and generate a rectal cancer analysis label.

Images