Position self-supervised colorectal CT image lesion identification and positioning method

Abstract

The invention discloses a position self-supervised colorectum CT image lesion identification and positioning method, which significantly reduces the workload of doctors through automatic image level classification and Patch level positioning, reduces the dependence on annotation data through self-supervised learning, and specifically comprises the steps of constructing a multi-modal CT scanning sequence data set, selecting and annotating key frames, preprocessing images, and identifying the lesion of the colorectum CT image. Constructing and training a colorectum CT image recognition and positioning network; the network comprises an image-level classification branch and a Patch-level positioning branch, and the Patch-level positioning branch enhances lesion area recognition through a lesion position self-supervision module; the method further comprises time sequence and modal consistency constraint so as to improve the accuracy of lesion area positioning. In addition, according to the method, through a mask correction loop strategy, the consistency of category labels and lesion positions is enhanced; as an auxiliary tool, the method can effectively assist doctors in surgical planning and treatment selection, and has important clinical application value.

Description

Technical Field

[0001] The present invention belongs to the field of medical image processing, and relates to a CT image automatic analysis and lesion location technology, and in particular to a position self-supervised colorectal CT image lesion recognition and location method. Background Art

[0002] Colorectal cancer (CRC) is one of the most common malignant tumors in the world. Its incidence rate has been increasing year by year in my country. According to statistics, it accounts for about 10% of all cancer cases and is the third most common cancer after lung cancer and gastric cancer. Since colorectal cancer often lacks obvious symptoms in the early stages, regular screening plays a vital role in early detection and treatment, which can not only improve the survival rate of patients, but also significantly improve the quality of life of patients.

[0003] In modern medicine, computed tomography (CT) technology has become a key tool for colorectal cancer screening and diagnosis because of its ability to provide high-resolution images and its non-invasive characteristics. However, traditional CT image analysis mainly relies on manual screening by radiologists, a process that is not only time-consuming and labor-intensive, but also highly dependent on the doctor's personal experience and subjective judgment, which may lead to inconsistencies and errors in diagnostic results. Therefore, improving the efficiency and accuracy of diagnosis has become an urgent problem to be solved.

[0004] In order to address this challenge, automated analysis methods for medical images have emerged and have gradually become a hot topic of research. In particular, the rapid development of deep learning technology has provided new possibilities for automated analysis of CT images and accurate location of lesions. In this field, researchers mainly focus on three aspects: classification, detection, and segmentation of CT images. Classification methods for CT images usually use two-dimensional or three-dimensional convolutional neural networks (CNNs) to analyze CT images to determine whether there are signs of colorectal cancer in the CT images. These methods can identify characteristic patterns related to colorectal cancer by learning a large amount of image data. For the detection and segmentation of CT images, deep learning models such as YOLO (You Only Look Once) and U-Net can more accurately locate the lesion area in the CT image. These deep learning models can identify the boundaries of the lesion area through training and segment it from the surrounding healthy tissue.

[0005] Although deep learning technology has shown great potential in the field of medical image processing, in order to achieve high-accuracy classification, detection and segmentation, a large number of finely labeled samples are usually required, which not only increases the clinical workload but also limits the application of deep learning technology in actual clinical settings.

[0006] Therefore, to overcome these limitations, researchers are exploring new algorithms and techniques to reduce the reliance on large amounts of labeled data and improve the clinical applicability of deep learning models. Summary of the invention

[0007] The technical problem to be solved by the present invention is to provide a method for colorectal CT image lesion recognition and positioning with position self-supervision, which only requires a small amount of finely labeled samples while ensuring that the position of the lesion area is accurately located after the colorectal CT image containing the lesion area is identified, which not only reduces the computational complexity but also reduces the labeling cost.

[0008] The technical solution adopted by the present invention to solve the above technical problems is: a method for recognizing and locating colorectal CT image lesions with position self-supervision, characterized by comprising the following steps:

[0009] Step 1: construct a data set, which contains three-modality colorectal CT scan sequences of several patients and several healthy individuals, and these sequences have been desensitized, wherein the three-modality colorectal CT scan sequences include plain CT scan sequences, venous phase CT scan sequences, and arterial phase CT scan sequences;

[0010] Step 2: Select the three modal colorectal CT scan sequences of some individual patients from the data set, and then annotate the plain scan slices containing the lesion area in each selected plain scan CT scan sequence. The specific process is as follows:

[0011] First, for a plain scan CT scan sequence, all plain scan slices containing lesion areas are selected, and these selected plain scan slices are continuous in the time domain; then, the first and last frames of plain scan slices are extracted from all the selected plain scan slices, and multiple frames of plain scan slices are randomly extracted from all the plain scan slices between the first and last frames, and these extracted plain scan slices are used as key frames, and the unextracted plain scan slices are used as non-key frames, wherein the number of frames of the plain scan slices randomly extracted from all the plain scan slices between the first and last frames is 10% to 20% of the number of frames of all the selected plain scan slices; then, the lesion area in each key frame is finely annotated, wherein the fine annotation is in the form of drawing a rectangular frame; finally, based on the annotated key frames, the position and size of the lesion area in each non-key frame are annotated by using linear difference to complete;

[0012] Step 3: Using the preset window width and window level, convert all plain scan slices contained in the plain scan CT scan sequences in the data set into 256-level plain scan grayscale images, convert all venous phase slices contained in the venous phase CT scan sequences in the data set into 256-level venous phase grayscale images, and convert all arterial phase slices contained in the arterial phase CT scan sequences in the data set into 256-level arterial phase grayscale images; then scale all plain scan grayscale images, venous phase grayscale images, and arterial phase grayscale images to W×H, where W represents width and H represents height; after scaling, all plain scan grayscale images, venous phase grayscale images, and arterial phase grayscale images constitute a pre-training set, and all labeled plain scan grayscale images and the corresponding venous phase grayscale images and arterial phase grayscale images constitute a training set;

[0013] Step 4: Construct a colorectal CT image recognition and localization network, which consists of an image-level classification branch and a patch-level localization branch. The image-level classification branch and the patch-level localization branch share a backbone network. The image-level classification branch also includes a fully connected layer. The patch-level localization branch also includes a patch-level classifier and a lesion location self-supervision module. The patch-level classifier consists of three sequentially connected convolutional blocks. The first and second convolutional blocks both contain a convolutional layer, a batch normalization layer, and a ReLU activation layer connected in sequence. The third convolutional block contains only one convolutional layer.

[0014] Step 5: Based on the pre-training set, pre-train the image-level classification branch in batches: each frame of plain scan grayscale image or venous phase grayscale image or arterial phase grayscale image in a batch is input into the backbone network, and the backbone network outputs a high-dimensional Patch feature vector corresponding to each frame. The Patch feature vector corresponding to each frame is input into the fully connected layer, and the fully connected layer outputs a probability value corresponding to each frame. The probability value corresponding to each frame is used to determine whether the frame contains a lesion area;

[0015] After a batch of training is completed, the classification loss is calculated and the network parameters of the image-level classification branch are updated through back-propagation;

[0016] After training for a preset number of epochs, the pre-training process ends and the model weights of the image-level classification branch are obtained;

[0017] Step 6: Based on the model weights of the image-level classification branch, the colorectal CT image recognition and positioning network is trained in batches based on the training set: a batch is set to contain multiple consecutive plain scan grayscale images from the same individual and the corresponding venous phase grayscale images and arterial phase grayscale images; the training is divided into two stages:

[0018] In the first stage, for the image-level classification branch, each frame in a batch is input into the backbone network, and the backbone network outputs a high-dimensional Patch feature vector corresponding to each frame. If the frame is a plain scan grayscale image, the Patch feature vector corresponding to the frame is input into the fully connected layer, and the fully connected layer outputs the probability value corresponding to the frame. The probability value corresponding to the frame is used to determine whether the frame contains a lesion area; if the frame is a venous phase grayscale image or an arterial phase grayscale image, no subsequent processing is performed;

[0019] Synchronously, for the patch-level positioning branch, each frame in the same batch is input into the backbone network, and the backbone network outputs a high-dimensional patch feature vector corresponding to each frame. The patch feature vector corresponding to each frame is input into the patch-level classifier, and the patch-level classifier outputs a grid patch corresponding to each frame. Each patch in the grid patch corresponding to each frame has a probability value, and the probability value of each patch is used to represent the lesion probability; the grid patch corresponding to each frame is input into the lesion location self-supervision module, and the implementation process of the lesion location self-supervision module is as follows:

[0020] For any frame of plain scan grayscale image input into the backbone network, if the frame is the first frame or the last frame of all the annotated plain scan grayscale images corresponding to an individual, the frame will not be processed; otherwise, the grid patch corresponding to the frame is defined as the reference grid patch, the grid patch corresponding to the previous plain scan grayscale image in the time domain of the frame is defined as the first auxiliary grid patch, the grid patch corresponding to the next plain scan grayscale image in the time domain of the frame is defined as the second auxiliary grid patch, the grid patch corresponding to the venous phase grayscale image corresponding to the frame is defined as the third auxiliary grid patch, and the grid patch corresponding to the arterial phase grayscale image corresponding to the frame is defined as the fourth auxiliary grid patch; then, the grid patches corresponding to the reference grid patch and the grid patches corresponding to the arterial phase grayscale image are respectively obtained from the reference grid patch and the grid patches corresponding to the arterial phase grayscale image. and the first to fourth auxiliary grid patches to find the patch with the highest probability of lesions; according to the deviations of the horizontal and vertical coordinates of the patch with the highest probability of lesions in the reference grid patch, the patch with the highest probability of lesions in the first auxiliary grid patch, and the patch with the highest probability of lesions in the second auxiliary grid patch, a rectangular area centered on the patch with the highest probability of lesions and containing all possible lesion areas is set in the reference grid patch; the areas in the first to fourth auxiliary grid patches that are at the same position as the rectangular area in the reference grid patch are used as the rectangular areas set therein; finally, the areas in the reference grid patch and the first to fourth auxiliary grid patches except the rectangular areas are used as the background areas;

[0021] In the first stage, after a batch of training is completed, the classification loss is calculated and the self-supervision loss is calculated;

[0022] In the second stage, the mask correction loop strategy is implemented: for any frame of plain scan grayscale image input to the backbone network, the grid patch corresponding to the frame is scaled to W×H; then the grid patch of size W×H corresponding to the frame is dot-multiplied with the matrix form of the frame to obtain the plain scan mask image of the lesion position mask; then the plain scan mask image is input into the backbone network, and the backbone network outputs a high-dimensional Patch feature vector corresponding to the plain scan mask image. The Patch feature vector corresponding to the plain scan mask image is input into the fully connected layer, and the fully connected layer outputs a probability value corresponding to the plain scan mask image;

[0023] In the second stage, after a batch of training is completed, the loop consistency loss is calculated;

[0024] Calculate the weighted sum of the classification loss in the first stage, the self-supervision loss in the first stage, and the loop consistency loss in the second stage as the total loss, and then update the network parameters of the colorectal CT image recognition and localization network through back propagation;

[0025] After training a preset number of epochs, the training process ends and the model weights of the colorectal CT image recognition and localization network are obtained;

[0026] Step 7: In the test phase, the plain scan CT scan sequence of any individual is used as the test sequence; then, the plain scan slices contained in the test sequence are converted into 256-level plain scan grayscale images using the same window width and window position as in step 3; then, all plain scan grayscale images are scaled to W×H; each frame of plain scan grayscale image is input into the colorectal CT image recognition and localization network that has obtained the model weights. The image-level classification branch in the colorectal CT image recognition and localization network outputs a probability value corresponding to each frame of plain scan grayscale image to realize the classification of whether each frame of plain scan grayscale image contains a lesion area. When a frame of plain scan grayscale image contains a lesion area, the patch-level localization branch outputs a grid patch corresponding to this frame of plain scan grayscale image, scales the grid patch corresponding to this frame of plain scan grayscale image to W×H, and performs dot multiplication between the grid patch of size W×H and the matrix form of this frame of plain scan grayscale image to obtain the position of the lesion area in this frame of plain scan grayscale image.

[0027] In step 1, the process of constructing the data set is:

[0028] Step 1.1: collecting three-modality colorectal CT scan sequences of more than 3,000 individuals, wherein the individuals include patients and healthy individuals, and the three-modality colorectal CT scan sequences include plain scan CT scan sequences, venous phase CT scan sequences, and arterial phase CT scan sequences;

[0029] Step 1.2: Clean all the collected colorectal CT scan sequences of the three modalities;

[0030] Step 1.3: All three modalities of colorectal CT scan sequences retained after cleaning were desensitized to remove personal information and form a dataset.

[0031] In step 1.2, the cleaning process is: for each individual's three-modality colorectal CT scan sequence, if the total number of frames of plain scan slices contained in the plain scan CT scan sequence, the total number of frames of venous phase slices contained in the venous phase CT scan sequence, and the total number of frames of arterial phase slices contained in the arterial phase CT scan sequence are equal, then the three-modality colorectal CT scan sequence of this individual is retained; otherwise, the three-modality colorectal CT scan sequence of this individual is cleaned.

[0032] In step 3, the preset window width is 400 and the preset window level is 60.

[0033] In step 4, the backbone network uses the ConvNeXtV2 network.

[0034] In step 5, the classification loss refers to the cross entropy loss of all frames in the batch determined by the image-level classification branch to contain the lesion area and all frames that do not contain the lesion area.

[0035] In step 5, 10 epochs are preset, the batch size batch_size is set to 48, the Adam optimizer is used, and the learning rate is set to 0.0001.

[0036] In step 6, the process of obtaining the rectangular area centered on the patch with the highest lesion probability in the reference grid Patch is as follows: calculating the deviations of the horizontal coordinates of the patch with the highest lesion probability in the reference grid Patch from the horizontal coordinates of the patch with the highest lesion probability in the first auxiliary grid Patch and the horizontal coordinates of the patch with the highest lesion probability in the second auxiliary grid Patch, and taking the maximum deviation of the two deviations as the horizontal coordinate deviation; similarly, calculating the deviations of the vertical coordinates of the patch with the highest lesion probability in the reference grid Patch from the vertical coordinates of the patch with the highest lesion probability in the first auxiliary grid Patch and the vertical coordinates of the patch with the highest lesion probability in the second auxiliary grid Patch, and taking the maximum deviation of the two deviations as the vertical coordinate deviation; and then, in the reference grid Patch, with the patch with the highest lesion probability as the center, setting a rectangular area with 2 times the horizontal coordinate deviation plus 1 patch in the horizontal coordinate direction and 2 times the vertical coordinate deviation plus 1 patch in the vertical coordinate direction.

[0037] In step 6, the classification loss in the first stage refers to the cross entropy loss of all frames in the batch that the image-level classification branch determines to contain the lesion area and all frames that do not contain the lesion area. The self-supervised loss in the first stage is the sum of the temporal consistency loss, the modal consistency loss and the weak supervision guidance strategy loss. The temporal consistency loss is the JS divergence loss of the background area in the reference grid patch corresponding to all the plain scan grayscale images in the batch and the background area in the corresponding first auxiliary grid patch, and the JS divergence loss of the background area in the reference grid patch corresponding to all the plain scan grayscale images in the batch and the background area in the corresponding second auxiliary grid patch. The modal consistency loss is the background area in the reference grid patch corresponding to all the plain scan grayscale images in the batch. The sum of the JS divergence losses of the background area in the reference grid patch corresponding to all plain scan grayscale images in the batch and the background area in the corresponding fourth auxiliary grid patch, the weakly supervised guided strategy loses the cross entropy loss of the rectangular area in the reference grid patch corresponding to all plain scan grayscale images in the batch and the lesion area in the plain scan grayscale image mapped to the reference grid patch, and the loop consistency loss in the second stage refers to the cross entropy loss of the probability values ​​corresponding to all plain scan mask images and 0; when calculating the total loss, the weight of the classification loss in the first stage is 1, the weight of the self-supervised loss in the first stage is 0.5, and the weight of the loop consistency loss in the second stage is 0.5.

[0038] In step 6, 10 epochs are preset, the batch size batch_size is set to 48, the Adam optimizer is used, and the learning rate is set to 0.0001.

[0039] Compared with the prior art, the advantages of the present invention are:

[0040] 1) The method of the present invention significantly reduces the workload of doctors in image analysis through automated image-level classification and patch-level positioning, and improves the efficiency of colorectal cancer screening and diagnosis; at the same time, it reduces the diagnostic inconsistency and errors caused by doctors' personal experience and subjective judgment. Without the need for a large amount of annotation, the patch-level positioning branch in the colorectal CT image recognition and positioning network constrains the temporal and modal consistency of the lesion position, and can accurately locate the position of the lesion area in the image containing the lesion area.

[0041] 2) Multiple modal CT scan sequences (plain scan, venous phase, and arterial phase) were used to train the colorectal CT image recognition and localization network, so that the trained colorectal CT image recognition and localization network can more accurately identify images containing lesion areas and locate the positions of lesion areas; CT images at different enhancement stages were considered, which helps to capture lesion features from different angles and may improve the sensitivity and specificity of lesion detection.

[0042] 3) By adopting a self-supervisory mechanism for lesion location, there is no need to label the venous and arterial phase slices, so that the colorectal CT image recognition and positioning network can be trained without a large amount of labeled data. For plain scan slices, by introducing key frames and non-key frames, as well as linear difference completion technology, the amount of data that requires fine labeling can be significantly reduced, thereby reducing labor costs and time costs, while reducing the clinical workload and improving the clinical applicability of the model.

[0043] 4) Considering the spatial continuity of the lesion position between consecutive frames under the same modality, the temporal consistency constraint is adopted; for plain scan and enhanced CT, the modality consistency constraint is adopted to improve the accuracy of locating the lesion area.

[0044] 5) The method of the present invention can be used as an auxiliary tool to help doctors make more accurate decisions in terms of surgical planning, treatment selection, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 The figure is a flowchart of the overall implementation process of the method of the present invention. DETAILED DESCRIPTION

[0046] The present invention is further described in detail below with reference to the accompanying drawings.

[0047] The present invention proposes a method for recognizing and locating colorectal CT image lesions with self-supervision, and its overall implementation flow chart is as follows: Figure 1 As shown, it includes the following steps:

[0048] Step 1: Construct a dataset, which contains three-modality colorectal CT scan sequences of several patients and several healthy individuals, and these sequences have been desensitized. The three-modality colorectal CT scan sequences include plain CT scan sequences, venous phase CT scan sequences, and arterial phase CT scan sequences. The patients have been judged as positive for colorectal cancer by experts based on clinical diagnosis results, and the venous phase CT scan sequences and arterial phase CT scan sequences are enhanced CT scan sequences.

[0049] In this embodiment, in step 1, the process of constructing the data set is:

[0050] Step 1.1: Collect three-modality colorectal CT scan sequences of more than 3,000 individuals, where the individuals include patients and healthy individuals, and the three-modality colorectal CT scan sequences include plain CT scan sequences, venous phase CT scan sequences, and arterial phase CT scan sequences. Here, since the neural network requires a large amount of data to achieve excellent performance, at least 3,000 individual colorectal CT scan sequences of three modalities should be collected. In the simulation experiment, three-modality colorectal CT scan sequences of 3,178 patient individuals and 887 healthy individuals were collected.

[0051] Step 1.2: Clean all the collected colorectal CT scan sequences of the three modalities.

[0052] Here, in step 1.2, the cleaning process is: for each individual's three-modality colorectal CT scan sequence, if the total number of frames of plain scan slices contained in the plain scan CT scan sequence, the total number of frames of venous phase slices contained in the venous phase CT scan sequence, and the total number of frames of arterial phase slices contained in the arterial phase CT scan sequence are equal, then the three-modality colorectal CT scan sequence of this individual is retained; otherwise, the three-modality colorectal CT scan sequence of this individual is cleaned.

[0053] Step 1.3: All three modalities of colorectal CT scan sequences retained after cleaning were desensitized to remove personal information and form a dataset.

[0054] Step 2: Select the three modal colorectal CT scan sequences of some individual patients (25% of the individual patients can be selected in the specific implementation) from the data set, and then annotate the plain scan slices containing the lesion area in each selected plain scan CT scan sequence. The specific process is as follows:

[0055] Firstly, for a plain scan CT scan sequence, all plain scan slices containing the lesion area are selected, and these selected plain scan slices are continuous in the time domain; then, the first and last frames of plain scan slices are extracted from all the selected plain scan slices, and multiple frames of plain scan slices are randomly extracted from all the plain scan slices between the first and last frames, and these extracted plain scan slices are used as key frames, and the unextracted plain scan slices are used as non-key frames, wherein the number of frames of the plain scan slices randomly extracted from all the plain scan slices between the first and last frames is 10% to 20% of the number of frames of all the selected plain scan slices; then, the lesion area in each key frame is finely marked, wherein the fine marking is in the form of drawing a rectangular frame; finally, based on the marked key frames, the position and size of the lesion area in each non-key frame are completed by using linear difference to mark.

[0056] Since the colorectum is a continuous lumen, only the key frames need to be directly annotated, and the non-key frames are annotated by using linear interpolation to complete the position and size of the lesion area. Therefore, only a small amount of fine annotation is required, which can significantly reduce the workload of fine annotation.

[0057] Step 3: Using the preset window width and window level, convert the plain scan slices contained in all plain scan CT scan sequences in the data set into 256-level plain scan grayscale images, convert the venous phase slices contained in all venous phase CT scan sequences in the data set into 256-level venous phase grayscale images, and convert the arterial phase slices contained in all arterial phase CT scan sequences in the data set into 256-level arterial phase grayscale images; then scale all plain scan grayscale images, venous phase grayscale images, and arterial phase grayscale images to W×H, where W represents width and H represents height, and in this embodiment, W=H=224; after scaling, all plain scan grayscale images, venous phase grayscale images, and arterial phase grayscale images constitute a pre-training set, and all annotated plain scan grayscale images and corresponding venous phase grayscale images and arterial phase grayscale images constitute a training set. Here, the consistency of image data and the standardization of model input are ensured by setting a unified window width and window level, as well as standardized scaling of images.

[0058] In this embodiment, in step 3, the preset window width is 400, and the preset window level is 60. Here, the window width and the window level are determined through a large number of experiments, which can ensure the effectiveness and reliability of the method of the present invention.

[0059] Step 4: Construct a colorectal CT image recognition and localization network, which consists of an image-level classification branch and a patch-level localization branch. The image-level classification branch and the patch-level localization branch share a backbone network. The image-level classification branch also includes a fully connected layer. The patch-level localization branch also includes a patch-level classifier and a lesion location self-supervision module. The patch-level classifier consists of three sequentially connected convolutional blocks. The first and second convolutional blocks both contain a sequentially connected convolutional layer (Conv2d), a batch normalization layer, and a batch normalization layer. The third convolutional block contains only one convolutional layer (Conv2d), the input of the convolutional layer in the first convolutional block is the input of the Patch-level classifier, the input of the convolutional layer in the second convolutional block receives the features output by the output of the ReLU activation layer in the first convolutional block, the input of the convolutional layer in the third convolutional block receives the features output by the output of the ReLU activation layer in the second convolutional block, and the output of the convolutional layer in the third convolutional block is the output of the Patch-level classifier. Here, the parameters of the fully connected layer and the parameters of the three convolutional layers will match the parameters of the backbone network.

[0060] In this embodiment, in step 4, the backbone network uses a convolutional neural network such as the ConvNeXtV2 network. Here, using an advanced convolutional neural network such as the ConvNeXtV2 network as the backbone network can improve the feature extraction capability and recognition accuracy of the model.

[0061] Step 5: Based on the pre-training set, pre-train the image-level classification branch by batch: each frame of the plain scan grayscale image or venous phase grayscale image or arterial phase grayscale image in a batch (the frames in this batch are random) is input into the backbone network F b In the backbone network F b Output a high-dimensional Patch feature vector corresponding to each frame, and the Patch feature vector corresponding to each frame is input into the fully connected layer F c In the fully connected layer F c A probability value corresponding to each frame is output, and the probability value corresponding to each frame is used to determine whether the frame contains a lesion area.

[0062] After a batch of training is completed, the classification loss is calculated and the network parameters of the image-level classification branch are updated through back-propagation.

[0063] After training a preset number of epochs, the pre-training process ends and the model weights of the image-level classification branch are obtained.

[0064] In this embodiment, in step 5, the classification loss refers to the cross entropy loss of all frames in the batch that the image-level classification branch determines to contain the lesion area and all frames that do not contain the lesion area; 10 epochs are preset, the batch size batch_size is set to 48, the Adam optimizer is used, and the learning rate is set to 0.0001.

[0065] Step 6: Based on the model weights of the image-level classification branch, the colorectal CT image recognition and positioning network is trained in batches based on the training set: a batch is set to contain multiple consecutive plain scan grayscale images from the same individual and the corresponding venous phase grayscale images and arterial phase grayscale images; the training is divided into two stages:

[0066] In the first stage, for the image-level classification branch, each frame in a batch is input to the backbone network F b In the backbone network F b Output a high-dimensional Patch feature vector corresponding to each frame. If the frame is a plain scan grayscale image, the Patch feature vector corresponding to the frame is input into the fully connected layer F c In the fully connected layer F c The probability value corresponding to the frame is output, and the probability value corresponding to the frame is used to determine whether the frame contains a lesion area; if the frame is a venous phase grayscale image or an arterial phase grayscale image, no subsequent processing is performed.

[0067] Synchronously, for the patch-level localization branch, each frame in the same batch is input to the backbone network F b In the backbone network F b Output a high-dimensional Patch feature vector corresponding to each frame, and the Patch feature vector corresponding to each frame is input into the Patch level classifier F pc In the Patch level classifier F pc Output a grid patch corresponding to each frame. Each patch in the grid patch corresponding to each frame has a probability value. The probability value of each patch is used to represent the lesion probability. The grid patch corresponding to each frame is input into the lesion location self-supervision module. The implementation process of the lesion location self-supervision module is as follows:

[0068] For the input backbone network F bFor any plain scan grayscale image of an individual, if the frame is the first or last frame of all the annotated plain scan grayscale images corresponding to an individual, the frame will not be processed; otherwise, the grid patch corresponding to the frame is defined as the reference grid patch, the grid patch corresponding to the previous plain scan grayscale image on the time domain of the frame is defined as the first auxiliary grid patch, the grid patch corresponding to the next plain scan grayscale image on the time domain of the frame is defined as the second auxiliary grid patch, the grid patch corresponding to the venous phase grayscale image corresponding to the frame is defined as the third auxiliary grid patch, and the grid patch corresponding to the arterial phase grayscale image corresponding to the frame is defined as the fourth auxiliary grid patch; then, the patch with the highest probability of lesions is found from the reference grid patch and the first to fourth auxiliary grid patches, and the five patches with the highest probability of lesions are denoted as P k n , P k n-1 , P k n+1 , P k a , P k v , according to the patch with the highest probability of lesion in the benchmark grid Patch (i.e., P k n ) and the patch with the highest probability of lesion in the first auxiliary grid patch (i.e., P k n-1 ) and the patch with the highest probability of lesions in the second auxiliary grid patch (i.e., P k n+1 ) of the horizontal and vertical coordinates, and set a patch with the highest probability of lesions (i.e., P k n ) as the center and including all possible lesion areas; then, the areas in the first to fourth auxiliary grid patches that are at the same position as the rectangular areas in the reference grid patches are used as rectangular areas set in them; finally, the areas except the rectangular areas in the reference grid patches and the first to fourth auxiliary grid patches are used as background areas.

[0069] In the first stage, after a batch of training is completed, the classification loss L is calculated cls , and calculate the self-supervised loss L con .

[0070] In the second stage, the mask correction loop strategy is implemented: for the input backbone network F bTake any plain scan grayscale image of the frame, scale the grid patch corresponding to the frame to W×H; then perform dot multiplication between the grid patch of size W×H corresponding to the frame and the matrix form of the frame to obtain the plain scan mask image of the lesion position mask; then input the plain scan mask image into the backbone network F b In the backbone network F b Output a high-dimensional Patch feature vector corresponding to the plain scan mask image, and the Patch feature vector corresponding to the plain scan mask image is input to the fully connected layer F c In the fully connected layer F c Output a probability value corresponding to the plain scan mask image.

[0071] In the second stage, after a batch of training is completed, the loop consistency loss L is calculated loop .

[0072] The weighted sum of the classification loss in the first stage, the self-supervision loss in the first stage, and the loop consistency loss in the second stage is calculated as the total loss, and then the network parameters of the colorectal CT image recognition and localization network are updated through back propagation.

[0073] After training a preset number of epochs, the training process ends and the model weights of the colorectal CT image recognition and positioning network are obtained.

[0074] In this embodiment, in step 6, the process of obtaining the rectangular area centered on the patch with the highest lesion probability in the reference grid Patch is as follows: calculating the deviations of the horizontal coordinates of the patch with the highest lesion probability in the reference grid Patch from the horizontal coordinates of the patch with the highest lesion probability in the first auxiliary grid Patch and the horizontal coordinates of the patch with the highest lesion probability in the second auxiliary grid Patch (i.e., the absolute value of the difference between the two), and taking the maximum deviation of the two deviations as the horizontal coordinate deviation; similarly, calculating the deviations of the vertical coordinates of the patch with the highest lesion probability in the reference grid Patch from the vertical coordinates of the patch with the highest lesion probability in the first auxiliary grid Patch and the vertical coordinates of the patch with the highest lesion probability in the second auxiliary grid Patch (i.e., the absolute value of the difference between the two), and taking the maximum deviation of the two deviations as the vertical coordinate deviation; and then, in the reference grid Patch, with the patch with the highest lesion probability as the center, setting a rectangular area with 2 times the horizontal coordinate deviation plus 1 patch in the horizontal coordinate direction and 2 times the vertical coordinate deviation plus 1 patch in the vertical coordinate direction.

[0075] In this embodiment, in step 6, the classification loss in the first stage refers to the cross entropy loss of all frames in the batch that the image-level classification branch determines to contain the lesion area and all frames that do not contain the lesion area. The self-supervised loss in the first stage is the sum of the temporal consistency loss, the modality consistency loss and the weakly supervised guidance strategy loss. The temporal consistency loss is the sum of the JS divergence loss of the background area in the reference grid patch corresponding to all the plain scan grayscale images in the batch and the background area in the corresponding first auxiliary grid patch, and the sum of the JS divergence loss of the background area in the reference grid patch corresponding to all the plain scan grayscale images in the batch and the background area in the corresponding second auxiliary grid patch. The modality consistency loss is the sum of the JS divergence loss of the background area in the reference grid patch corresponding to all the plain scan grayscale images in the batch and the background area in the corresponding third auxiliary grid patch. The sum of the JS divergence loss of the background area in the benchmark grid patch corresponding to all the plain scan grayscale images in the batch and the background area in the corresponding fourth auxiliary grid patch, the weak supervision guidance strategy loss is the cross entropy loss of the rectangular area in the benchmark grid patch corresponding to all the plain scan grayscale images in the batch and the lesion area in the plain scan grayscale image mapped to the benchmark grid patch, and the loop consistency loss in the second stage refers to the cross entropy loss of the probability value corresponding to all plain scan mask images and 0; when calculating the total loss, the weight of the classification loss in the first stage is 1, the weight of the self-supervision loss in the first stage is 0.5, and the weight of the loop consistency loss in the second stage is 0.5; 10 epochs are preset, the batch size batch_size is set to 48, the Adam optimizer is used, and the learning rate is set to 0.0001.

[0076] Step 7: In the test phase, the plain scan CT scan sequence of any individual is used as the test sequence; then, the plain scan slices contained in the test sequence are converted into 256-level plain scan grayscale images using the same window width and window position as in step 3; then, all plain scan grayscale images are scaled to W×H; each frame of plain scan grayscale image is input into the colorectal CT image recognition and localization network that has obtained the model weights. The image-level classification branch in the colorectal CT image recognition and localization network outputs a probability value corresponding to each frame of plain scan grayscale image to realize the classification of whether each frame of plain scan grayscale image contains a lesion area. When a frame of plain scan grayscale image contains a lesion area, the patch-level localization branch outputs a grid patch corresponding to this frame of plain scan grayscale image, scales the grid patch corresponding to this frame of plain scan grayscale image to W×H, and performs dot multiplication between the grid patch of size W×H and the matrix form of this frame of plain scan grayscale image to obtain the position of the lesion area in this frame of plain scan grayscale image.

[0077] If the number of frames of the plain scan grayscale image containing the lesion area is greater than 7 frames (this data is obtained through a large number of experiments), the test sequence is determined to be a colorectal cancer CT scan sequence.

[0078] In order to further illustrate the feasibility and effectiveness of the method of the present invention, a simulation experiment was carried out on the method of the present invention.

[0079] A colorectal CT scan dataset was collected from three medical centers. The three modalities of colorectal CT scan sequences for each individual were plain scan, venous phase CT scan, and arterial phase CT scan. The colorectal CT scan dataset contained three modalities of colorectal CT scan sequences of 3178 colorectal cancer patients and 887 healthy individuals (the number of slices of each modality was equal). The pre-training set contained three modalities of colorectal CT scan sequences of 3028 colorectal cancer patients and 737 healthy individuals (a total of 75642 slices containing lesions and 64182 slices not containing lesions, including plain slices, venous phase slices, and arterial phase slices). Only 6304 plain scan slices containing lesions were used (accounting for 8.3% of the total), and these plain scan slices were annotated with lesions to provide supervision information. The test set contains plain scan CT scan sequences of 150 colorectal cancer patients and 150 healthy individuals (a total of 1,200 plain scan slices containing lesion areas and 2,786 plain scan slices not containing lesion areas), which are used to evaluate the effectiveness of the method of the present invention.

[0080] In the simulation experiment, the methods used for comparison include ResNet50, MobileNetV3-L, RegNetY-128G, EfficientNetV2-L, ConvNeXt-L and ConvNeXtV2-L methods, which are currently the most advanced CNN-based image classification methods. The ResNet50 method was published in He K, Zhang X, Ren S, et al. Deep residual learning for image recognition [C] / / Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 770-778. (Deep residual learning for image recognition, IEEE Computer Vision and Pattern Recognition Conference Proceedings), the MobileNetV3-L method was published in Howard A, Sandler M, Chu G, et al. Searching for MobileNetV3 [C] / / Proceedings of the IEEE / CVF international conference on computer vision. 2019: 1314-1324. (Searching for Mobilenetv3, IEEE / CVF International Conference on Computer Vision Proceedings), and the RegNetY-128G method was published in Radosavovic I, Kosaraju RP, Girshick R, et al. Designing network design spaces [C] / / Proceedings of the IEEE / CVF conference on computer vision and pattern recognition. recognition.2020:10428-10436.(Construction of network design space, Proceedings of IEEE / CVF Conference on Computer Vision and Pattern Recognition), the EfficientNetV2 method is published in Tan M, Le Q. EfficientNetV2: Smaller models and faster training[C] / / International conference on machine learning. PMLR, 2021:10096-10106.(EfficientNetV2: Smaller models and faster training, International Conference on Machine Learning), and the ConvNeXt-L method is published in Liu Z, Mao H, Wu CY, et al.A ConvNet for the 2020s[C] / / Proceedings of the IEEE / CVF conference on computer vision and pattern recognition.2022:11976-11986.(ConvNet for the 2020s, IEEE / CVF Conference on Computer Vision and Pattern Recognition) has been made public, and the ConvNeXtV2-L method has been made public in Woo S, Debnath S, Hu R, et al.ConvNeXtV2:Co-designing and scaling convnets with masked autoencoders[C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition.2023:16133-16142.(ConvNeXt V2: Co-designing and scaling convolutional neural networks with masked autoencoders, IEEE / CVF Conference on Computer Vision and Pattern Recognition) has been made public. .

[0081] In order to verify the effectiveness of the method of the present invention, ResNet50, MobileNetV3-L, RegNetY-128G, EfficientNetV2-L, ConvNeXt-L and ConvNeXtV2-L were used as backbone networks, respectively, and an existing positioning branch was added. The existing positioning branch outputs 4 values, which respectively represent the horizontal and vertical coordinates of the upper left corner and the horizontal and vertical coordinates of the lower right corner of the positioning target area. The fully supervised annotated data of the colorectal CT scan sequences of the three modalities were used for training. The performance comparison of the method of the present invention in plain scan slice level classification and positioning is shown in Table 1.

[0082] Table 1 Classification and localization performance at the plain scan slice level

[0083]

[0084] As can be seen from Table 1, it is obvious that the proposed method performs well in the slice-level classification task. This is mainly because the proposed lesion location self-supervision module and mask correction loop strategy enhance the ability to distinguish between colorectal cancer samples and healthy samples. For the localization of lesion locations, considering that the network is not designed for regression, direct comparison of IoU is not appropriate. Therefore, on the common target localization evaluation indicators, the Patch level IoU is evaluated. Specifically, the original image is divided into grid blocks, and the Patch overlap rate of the predicted box and the actual box is calculated. Therefore, P-precision, P-recall, and P-Iou are used to evaluate the localization performance. The proposed method achieves the best performance in terms of P-precision and P-IoU, but performs poorly in terms of P-recall, which is due to the lack of extensive fine annotation guidance. The results show that using only 8.3% of the annotations, the proposed method achieves promising results compared to those methods that require a large number of annotations.

[0085] Table 2 shows the classification performance at the individual level. The classification performance uses the threshold method to analyze the entire colorectal CT scan sequence. For an individual test sequence, if the number of frames of the plain scan grayscale image containing the lesion area is greater than 7 frames, then the test sequence is determined to be a colorectal cancer CT scan sequence, and the individual is diagnosed as a colorectal cancer patient.

[0086] Table 2 Classification performance at individual level

[0087] method Accuracy Accuracy Recall Specificity F1-score ResNet50 84.00％ 82.28％ 86.67％ 81.33％ 84.42％ MobileNetV3-L 83.33％ 82.89％ 84.00％ 82.67％ 83.44％ RegNetY-128G 84.00％ 82.28％ 86.67％ 81.33％ 84.42％ EfficientNetV2 83.33％ 80.49％ 88.00％ 78.67％ 84.08％ ConvNeXt-L 85.33％ 84.87％ 86.00％ 84.67％ 85.43％ ConvNeXtV2-L 85.67％ 85.91％ 85.33％ 86.00％ 85.62％ Method of the present invention 87.00％ 86.27％ 88.00％ 86.00％ 87.13％

[0088] It can be observed from Table 2 that the method of the present invention once again exhibits superior performance. In clinical diagnosis, for the entire scanning sequence of the patient, the model can more effectively distinguish between lesions and healthy slices, further verifying the robustness and reliability of the method of the present invention.

[0089] The contents described in the embodiments of this specification are merely an enumeration of the implementation forms of the inventive concept. The protection scope of the present invention should not be regarded as limited to the specific forms described in the embodiments. The protection scope of the present invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. A method for position self-supervised colorectal CT image lesion recognition and localization, characterized in that The following steps are involved: Step 1: construct a data set, which contains three-modality colorectal CT scan sequences of several patients and several healthy individuals, and these sequences have been desensitized, wherein the three-modality colorectal CT scan sequences include plain CT scan sequences, venous phase CT scan sequences, and arterial phase CT scan sequences; Step 2: Select the three modal colorectal CT scan sequences of some individual patients from the data set, and then annotate the plain scan slices containing the lesion area in each selected plain scan CT scan sequence. The specific process is as follows: First, for a plain scan CT scan sequence, all plain scan slices containing lesion areas are selected, and these selected plain scan slices are continuous in the time domain; then, the first and last frames of plain scan slices are extracted from all the selected plain scan slices, and multiple frames of plain scan slices are randomly extracted from all the plain scan slices between the first and last frames, and these extracted plain scan slices are used as key frames, and the unextracted plain scan slices are used as non-key frames, wherein the number of frames of the plain scan slices randomly extracted from all the plain scan slices between the first and last frames is 10% to 20% of the number of frames of all the selected plain scan slices; then, the lesion area in each key frame is finely annotated, wherein the fine annotation is in the form of drawing a rectangular frame; finally, based on the annotated key frames, the position and size of the lesion area in each non-key frame are annotated by using linear difference to complete; Step 3: Using the preset window width and window level, convert all plain scan slices contained in the plain scan CT scan sequences in the data set into 256-level plain scan grayscale images, convert all venous phase slices contained in the venous phase CT scan sequences in the data set into 256-level venous phase grayscale images, and convert all arterial phase slices contained in the arterial phase CT scan sequences in the data set into 256-level arterial phase grayscale images; then scale all plain scan grayscale images, venous phase grayscale images, and arterial phase grayscale images to W×H, where W represents width and H represents height; after scaling, all plain scan grayscale images, venous phase grayscale images, and arterial phase grayscale images constitute a pre-training set, and all labeled plain scan grayscale images and the corresponding venous phase grayscale images and arterial phase grayscale images constitute a training set; Step 4: Construct a colorectal CT image recognition and localization network, which consists of an image-level classification branch and a patch-level localization branch. The image-level classification branch and the patch-level localization branch share a backbone network. The image-level classification branch also includes a fully connected layer. The patch-level localization branch also includes a patch-level classifier and a lesion location self-supervision module. The patch-level classifier consists of three sequentially connected convolutional blocks. The first and second convolutional blocks both contain a convolutional layer, a batch normalization layer, and a ReLU activation layer connected in sequence. The third convolutional block contains only one convolutional layer. Step 5: Based on the pre-training set, pre-train the image-level classification branch in batches: each frame of plain scan grayscale image or venous phase grayscale image or arterial phase grayscale image in a batch is input into the backbone network, and the backbone network outputs a high-dimensional Patch feature vector corresponding to each frame. The Patch feature vector corresponding to each frame is input into the fully connected layer, and the fully connected layer outputs a probability value corresponding to each frame. The probability value corresponding to each frame is used to determine whether the frame contains a lesion area; After a batch of training is completed, the classification loss is calculated and the network parameters of the image-level classification branch are updated through back-propagation; After training for a preset number of epochs, the pre-training process ends and the model weights of the image-level classification branch are obtained; Step 6: Based on the model weights of the image-level classification branch, the colorectal CT image recognition and positioning network is trained in batches based on the training set: a batch is set to contain multiple consecutive plain scan grayscale images from the same individual and the corresponding venous phase grayscale images and arterial phase grayscale images; the training is divided into two stages: In the first stage, for the image-level classification branch, each frame in a batch is input into the backbone network, and the backbone network outputs a high-dimensional Patch feature vector corresponding to each frame. If the frame is a plain scan grayscale image, the Patch feature vector corresponding to the frame is input into the fully connected layer, and the fully connected layer outputs the probability value corresponding to the frame. The probability value corresponding to the frame is used to determine whether the frame contains a lesion area; if the frame is a venous phase grayscale image or an arterial phase grayscale image, no subsequent processing is performed; Synchronously, for the patch-level positioning branch, each frame in the same batch is input into the backbone network, and the backbone network outputs a high-dimensional patch feature vector corresponding to each frame. The patch feature vector corresponding to each frame is input into the patch-level classifier, and the patch-level classifier outputs a grid patch corresponding to each frame. Each patch in the grid patch corresponding to each frame has a probability value, and the probability value of each patch is used to represent the lesion probability; the grid patch corresponding to each frame is input into the lesion location self-supervision module, and the implementation process of the lesion location self-supervision module is as follows: For any frame of plain scan grayscale image input into the backbone network, if the frame is the first frame or the last frame of all the annotated plain scan grayscale images corresponding to an individual, the frame will not be processed; otherwise, the grid patch corresponding to the frame is defined as the reference grid patch, the grid patch corresponding to the previous plain scan grayscale image in the time domain of the frame is defined as the first auxiliary grid patch, the grid patch corresponding to the next plain scan grayscale image in the time domain of the frame is defined as the second auxiliary grid patch, the grid patch corresponding to the venous phase grayscale image corresponding to the frame is defined as the third auxiliary grid patch, and the grid patch corresponding to the arterial phase grayscale image corresponding to the frame is defined as the fourth auxiliary grid patch; then, the grid patches corresponding to the reference grid patch and the grid patches corresponding to the arterial phase grayscale image are respectively obtained from the reference grid patch and the grid patches corresponding to the arterial phase grayscale image. and the first to fourth auxiliary grid patches to find the patch with the highest probability of lesions; according to the deviations of the horizontal and vertical coordinates of the patch with the highest probability of lesions in the reference grid patch, the patch with the highest probability of lesions in the first auxiliary grid patch, and the patch with the highest probability of lesions in the second auxiliary grid patch, a rectangular area centered on the patch with the highest probability of lesions and containing all possible lesion areas is set in the reference grid patch; the areas in the first to fourth auxiliary grid patches that are at the same position as the rectangular area in the reference grid patch are used as the rectangular areas set therein; finally, the areas in the reference grid patch and the first to fourth auxiliary grid patches except the rectangular areas are used as the background areas; In the first stage, after a batch of training is completed, the classification loss is calculated and the self-supervision loss is calculated; In the second stage, the mask correction loop strategy is implemented: for any frame of plain scan grayscale image input to the backbone network, the grid patch corresponding to the frame is scaled to W×H; then the grid patch of size W×H corresponding to the frame is dot-multiplied with the matrix form of the frame to obtain the plain scan mask image of the lesion position mask; then the plain scan mask image is input into the backbone network, and the backbone network outputs a high-dimensional Patch feature vector corresponding to the plain scan mask image. The Patch feature vector corresponding to the plain scan mask image is input into the fully connected layer, and the fully connected layer outputs a probability value corresponding to the plain scan mask image; In the second stage, after a batch of training is completed, the loop consistency loss is calculated; Calculate the weighted sum of the classification loss in the first stage, the self-supervision loss in the first stage, and the loop consistency loss in the second stage as the total loss, and then update the network parameters of the colorectal CT image recognition and localization network through back propagation; After training a preset number of epochs, the training process ends and the model weights of the colorectal CT image recognition and localization network are obtained; Step 7: In the test phase, the plain scan CT scan sequence of any individual is used as the test sequence; then, the plain scan slices contained in the test sequence are converted into 256-level plain scan grayscale images using the same window width and window position as in step 3; then, all plain scan grayscale images are scaled to W×H; each frame of plain scan grayscale image is input into the colorectal CT image recognition and localization network that has obtained the model weights. The image-level classification branch in the colorectal CT image recognition and localization network outputs a probability value corresponding to each frame of plain scan grayscale image to realize the classification of whether each frame of plain scan grayscale image contains a lesion area. When a frame of plain scan grayscale image contains a lesion area, the patch-level localization branch outputs a grid patch corresponding to this frame of plain scan grayscale image, scales the grid patch corresponding to this frame of plain scan grayscale image to W×H, and performs dot multiplication between the grid patch of size W×H and the matrix form of this frame of plain scan grayscale image to obtain the position of the lesion area in this frame of plain scan grayscale image.

2. A method for colorectal CT image lesion recognition and positioning based on position self-supervision according to claim 1, characterized in that In step 1, the process of constructing the data set is: Step 1.1: collecting three-modality colorectal CT scan sequences of more than 3,000 individuals, wherein the individuals include patients and healthy individuals, and the three-modality colorectal CT scan sequences include plain scan CT scan sequences, venous phase CT scan sequences, and arterial phase CT scan sequences; Step 1.2: Clean all the collected colorectal CT scan sequences of the three modalities; Step 1.3: All three modalities of colorectal CT scan sequences retained after cleaning were desensitized to remove personal information and form a dataset.

3. The method for position self-supervised colorectal CT image lesion recognition and positioning according to claim 2, characterized in that In step 1.2, the cleaning process is: for each individual's three-modality colorectal CT scan sequence, if the total number of frames of plain scan slices contained in the plain scan CT scan sequence, the total number of frames of venous phase slices contained in the venous phase CT scan sequence, and the total number of frames of arterial phase slices contained in the arterial phase CT scan sequence are equal, then the three-modality colorectal CT scan sequence of this individual is retained; otherwise, the three-modality colorectal CT scan sequence of this individual is cleaned.

4. The method for position self-supervised colorectal CT image lesion recognition and positioning according to claim 1, characterized in that In step 3, the preset window width is 400 and the preset window level is 60.

5. The method for position self-supervised colorectal CT image lesion recognition and positioning according to claim 1, characterized in that In step 4, the backbone network uses the ConvNeXtV2 network.

6. The method for position self-supervised colorectal CT image lesion recognition and positioning according to claim 1, characterized in that In step 5, the classification loss refers to the cross entropy loss of all frames in the batch determined by the image-level classification branch to contain the lesion area and all frames that do not contain the lesion area.

7. The method for recognizing and locating colorectal CT image lesions with position self-supervision according to claim 1, characterized in that In step 5, 10 epochs are preset, the batch size batch_size is set to 48, the Adam optimizer is used, and the learning rate is set to 0.0001.

8. The method for position self-supervised colorectal CT image lesion recognition and positioning according to claim 1, characterized in that In step 6, the process of obtaining the rectangular area centered on the patch with the highest lesion probability in the reference grid Patch is as follows: calculating the deviations of the abscissa of the patch with the highest lesion probability in the reference grid Patch from the abscissa of the patch with the highest lesion probability in the first auxiliary grid Patch and the abscissa of the patch with the highest lesion probability in the second auxiliary grid Patch, and taking the maximum deviation of the two deviations as the abscissa deviation; Similarly, calculate the deviations of the ordinate of the patch with the highest lesion probability in the reference grid Patch from the ordinates of the patch with the highest lesion probability in the first auxiliary grid Patch and the ordinate of the patch with the highest lesion probability in the second auxiliary grid Patch, and take the maximum deviation of the two deviations as the ordinate deviation. Then, in the reference grid Patch, with the patch with the highest lesion probability as the center, set a rectangular area with 2 times the horizontal coordinate deviation plus 1 patch in the horizontal direction and 2 times the vertical coordinate deviation plus 1 patch in the vertical direction.

9. The method for position self-supervised colorectal CT image lesion recognition and positioning according to claim 1, characterized in that In step 6, the classification loss in the first stage refers to the cross entropy loss of all frames in the batch that the image-level classification branch determines to contain the lesion area and all frames that do not contain the lesion area. The self-supervised loss in the first stage is the sum of the temporal consistency loss, the modal consistency loss and the weak supervision guidance strategy loss. The temporal consistency loss is the JS divergence loss of the background area in the reference grid patch corresponding to all the plain scan grayscale images in the batch and the background area in the corresponding first auxiliary grid patch, and the JS divergence loss of the background area in the reference grid patch corresponding to all the plain scan grayscale images in the batch and the background area in the corresponding second auxiliary grid patch. The modal consistency loss is the background area in the reference grid patch corresponding to all the plain scan grayscale images in the batch. The sum of the JS divergence losses of the background area in the reference grid patch corresponding to all plain scan grayscale images in the batch and the background area in the corresponding fourth auxiliary grid patch, the weakly supervised guided strategy loses the cross entropy loss of the rectangular area in the reference grid patch corresponding to all plain scan grayscale images in the batch and the lesion area in the plain scan grayscale image mapped to the reference grid patch, and the loop consistency loss in the second stage refers to the cross entropy loss of the probability values ​​corresponding to all plain scan mask images and 0; when calculating the total loss, the weight of the classification loss in the first stage is 1, the weight of the self-supervised loss in the first stage is 0.5, and the weight of the loop consistency loss in the second stage is 0.

5.

10. The method for position self-supervised colorectal CT image lesion recognition and positioning according to claim 1, characterized in that In step 6, 10 epochs are preset, the batch size batch_size is set to 48, the Adam optimizer is used, and the learning rate is set to 0.0001.

Images