A colorectal cancer staging algorithm and system based on a self-supervised learning mode

Abstract

The application provides a colorectal cancer staging algorithm and system based on a self-supervised learning mode, which can judge the severity of colorectal cancer without using labeled samples. First, image transformation and low-level processing are performed on the data set to generate different groups of new data sets. The different groups of data sets are input into different encoder networks to obtain feature outputs, and the image information is saved through the conversion of the conditional attention mechanism and the cross-model hybrid maximum. Finally, the model is trained and optimized by merging and calculating the contrast loss and the reconstruction loss. The application can effectively classify the severity of colorectal cancer without a labeled training set by learning the similarity and difference of medical images to construct features, and has a wide application prospect.

Description

Technical Field

[0001] This invention belongs to the field of medical imaging, specifically relating to a colorectal cancer staging algorithm and system based on self-supervised learning. Background Technology

[0002] Currently, training deep neural networks typically requires large amounts of manually labeled data. This requirement is easily met in natural images, as labor costs and labeling difficulties are acceptable. However, in medical image analysis, reliable medical annotations usually come from expert diagnoses, which are difficult to obtain given the scarcity of the target disease, patient privacy, and limited medical resources. To address these issues, self-supervised learning, as a method for learning medical image representations without manual annotation, has been widely adopted.

[0003] Contrastive representation learning is now widely used and has achieved significant success in medical image analysis. The goal of contrastive learning is to learn invariant representations by contrasting pairs of medical images, which can be seen as a way to preserve maximum information implicitly. Nevertheless, explicitly preserving more information, in addition to contrastive loss, is complementary and beneficial. The most intuitive solution currently is to reconstruct the original input using the learned representation. However, directly adding a standard reconstruction branch for recovering the original input does not significantly improve the learned representation. Summary of the Invention

[0004] This invention designs an image staging algorithm for colorectal cancer based on self-supervised learning, which classifies the severity of colorectal cancer using unlabeled samples and through self-supervised learning.

[0005] To achieve the above objectives, the technical solution proposed by this invention is as follows:

[0006] A novel fast medical image segmentation algorithm based on small sample sizes includes the following steps:

[0007] Step 1, Preserve Contrast Representation Learning. By reconstructing different contexts, more information is encoded into the representations learned from the contrastive loss.

[0008] Step 2, Transformation Conditional Attention Mechanism. The transformed vectors are encoded into high-level representations based on the attention mechanism, forcing the encoder to retain more information from the learned representations.

[0009] Step 3, cross-model fusion. Blending feature representations enables more diverse image restoration.

[0010] Step 4, Model Update. A fully connected layer and a softmax function are added to the end of the algorithm network, and a loss function is designed to update the model training parameters.

[0011] Furthermore, step 1 specifically includes:

[0012] Contrast representation learning comprises three distinct encoders and a shared decoder. The encoders and decoder are connected via a U-Net-like architecture. First, an exponential moving average is applied to the parameters of the standard encoder to generate a momentum encoder. Then, for each input, the representations (feature maps) from the two encoders are fused across models to construct a hybrid encoder. Given a batch of images... First, random cropping, random flipping, and random rotation are applied to generate three batches of images for three different encoders. and Then, low-level processing operations, including inpainting, expainting, and Gaussian blurring, are applied to each batch of images to generate the final input for different encoders. In each training step, three sets of transformations (including flips and rotations) are randomly generated: T o T m and T h This is then encoded into the last convolutional layer of each encoder. The ground truth objective of the MSE (mean squared error) loss in image reconstruction is... and This corresponds to different encoders. To implement contrastive learning in the algorithm, past image representations are stored in a queue, and then contrastive loss is applied to positive and negative image pairs.

[0013] Furthermore, step 2 specifically involves:

[0014] For each input, the indicator vector contains a combination of different transformations. Each component contains an indicator function (1 or 0) indicating whether a particular transformation is applied. To encode the indicator vector into a high-level feature map, a transformation-conditional attention mechanism is employed, which assumes that different channels of the feature map may have different effects on the reconstruction result. Furthermore, the transformation-conditional attention module is applied only to the last convolutional layer (before the fully connected layer) of each encoder.

[0015] First, the indicator vector vec(T) is passed to two fully connected (FC) layers to produce a vector f. p ∈R C×1 Simultaneously, global average pooling is applied to the high-level feature map F of each encoder. l ∈R C×D×H×W Obtain vector f l ∈R C×1 , where l represents the layer index. Then calculate f. p and f l The outer product M. Next, M is flattened and passed to another fully connected layer to obtain f. q In order to perform a scale re-adjustment, further shift to f qAdding a sigmoid function gives f w Finally, in F l and f w Channel multiplication is used between them, and a convolutional layer with a kernel size of 3 is added.

[0016] Furthermore, step 3 specifically involves:

[0017] Cross-model fusion shuffles these feature representations, constructing a new hybrid encoder by mixing feature maps from two different models.

[0018] Therefore, the reconstruction target of the hybrid encoder is the hybrid input. For network feature maps, use Let represent the feature map of the i-th layer of a normal encoder, where i ∈ {1,…,l}. Similarly... and These represent feature maps at the same location in the momentum encoder and hybrid encoder, respectively. Adding a shared encoder allows for direct use. To rebuild

[0019] Furthermore, step 4 specifically involves:

[0020] Use a queue K = {k1, ..., k N The queue K stores past features for comparison; its length is N. In contrastive learning, all features in queue K are treated as negative samples. o (·) and g m (·) represent the projections of a conventional encoder and a momentum encoder, respectively. g o (·) and g m (·) contains global equal pooling and two fully connected (FC) layers respectively. After each training iteration, and They are pushed into K as negative samples for further comparison.

[0021] For different reconstruction contexts, mean squared error (MSE) is used as the default reconstruction loss. Formally, the shared decoder network is represented as D. θ Given that the entire network has a similar architecture to U-Net, the input to the decoder should be a multi-layer feature map F. {o,m,h} The comparison loss L c and reconstruction loss L p The loss function is obtained by merging the parameters proportionally. For the network parameters, the parameters in the ordinary encoder and the momentum encoder are represented as θ. o and θ m θ is updated using the exponential moving average (EMA) factor β. m .

[0022] A colorectal cancer staging system based on self-supervised learning, used to implement the colorectal cancer staging algorithm based on self-supervised learning as described above, the system comprising:

[0023] The first module is set to preserve contrastive representation learning; by reconstructing different contexts, more information is encoded into the representations learned from the contrastive loss.

[0024] The second module is configured to transform the conditional attention mechanism; the transformation vector is encoded into a high-level representation according to the attention mechanism, which forces the encoder to retain information in the learned representation;

[0025] The third module is configured for cross-model blending; the blended features represent image restoration.

[0026] The fourth module is set up for model updates; a fully connected layer and a softmax function are added at the end of the algorithm network, and a loss function is designed to update the model training parameters.

[0027] The beneficial effects of this invention are as follows: This invention proposes a colorectal cancer staging algorithm and system based on self-supervised learning, which can determine the severity of colorectal cancer without the need for labeled samples. By learning to encode the similarity and differences of medical images to construct features, it can effectively classify the severity of colorectal cancer without labeled training sets, and has broad application prospects. Attached Figure Description

[0028] Figure 1 This is a conceptual diagram of a method according to an embodiment of the present invention;

[0029] Figure 2 This is a schematic diagram of a preserved contrastive representation learning framework according to an embodiment of the present invention;

[0030] Figure 3 This is a schematic diagram of the transition conditional attention mechanism according to an embodiment of the present invention. Detailed Implementation

[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] Example 1

[0033] This invention discloses a colorectal cancer staging algorithm based on self-supervised learning. Based on the principle of maximizing image information preservation, it employs preservation learning to reconstruct different image contexts and learns self-supervised medical representations through preservation-contrast representation learning. Specifically, it includes the following steps:

[0034] Step 1: Preserve contrastive representation learning.

[0035] like Figure 2 As shown, the contrastive representation learning method comprises three distinct encoders and a shared decoder. The encoders and decoders are connected via a U-Net-like architecture. First, an exponential moving average is applied to the parameters of the ordinary encoder to generate a momentum encoder. Then, for each input, the representations (feature maps) of the two encoders are fused across models to construct a hybrid encoder. Given a batch of images x, random cropping, random flipping, and random rotation are first applied to generate three batches of images for the three different encoders. and Then, low-level processing operations, including inpainting, expainting, and Gaussian blurring, are applied to each batch of images to generate the final input for different encoders. In each training step, three sets of transformations (including flips and rotations) are randomly generated: T o , and This is then encoded into the last convolutional layer of each encoder. The ground truth objective of the MSE (mean squared error) loss in image reconstruction is... and This corresponds to different encoders. To implement contrastive learning in the algorithm, past image representations are stored in a queue, and then contrastive loss is applied to positive and negative image pairs.

[0036] Step 2: Switch to conditional attention mechanism.

[0037] like Figure 3 As shown, for each input, the indicator vector contains a combination of different transformations. Specifically, given a 3D input (CT and MRI scans), the indicator vector has 7 components representing different transformation strategies. For a 2D input (e.g., X-ray), the number of transformations is reduced to 6, where there is no random flipping of the Z-axis. Each component contains an indicator function (1 or 0) indicating whether a specific transformation is applied. To encode the indicator vector into a high-level feature map, a transformation-conditional attention mechanism is employed, which assumes that different channels of the feature map may have different effects on the reconstruction result. It is worth noting here that the transformation-conditional attention module is only applied to the last convolutional layer (before the FC layer) of each encoder.

[0038] First, the indicator vector It is passed to two fully connected (FC) layers, producing a vector f.p ∈R C×1 Simultaneously, global average pooling is applied to the high-level feature map F of each encoder. l ∈R C×D×H×W Obtain vector f l ∈R C×1 , where l represents the layer index. Then calculate f. p and f l The outer product M:

[0039]

[0040] Where M∈R C×C .

[0041] Next, flatten M and pass it to another fully connected layer:

[0042] f q =ReLU(W θ flat.(M))

[0043] Among them W θ ∈R C×C2 This represents the weight parameters of the FC layer.

[0044] In order to perform the rescale change, further towards f q Add a sigmoid function:

[0045] f w =sigmoid(f q )

[0046] Where f w ∈R C×1×1×1 .

[0047] Finally, in F l and f w Channel multiplication is used between them, and a convolutional layer with a kernel size of 3 is added:

[0048] F l+1 =conv(F l ⊙f w )

[0049] Where F l+1 ∈R C×D×H×W .

[0050] Step 3: Cross-model blending.

[0051] Cross-model fusion shuffles these feature representations to achieve more diverse image restorations. A new hybrid encoder is constructed by merging feature maps from two different models.

[0052] Therefore, the reconstruction target of the hybrid encoder is the hybrid input. In fact, for each training iteration, the mixed input Input from a standard encoder and momentum encoder Input synthesis:

[0053]

[0054] Where λ ~ Beta(α,α), and α is a hyperparameter.

[0055] For network feature maps, use Let represent the feature map of the i-th layer of a normal encoder, where i ∈ {1,…,l}. Similarly... and These represent feature maps at the same location in the momentum encoder and the hybrid encoder, respectively. Therefore, the process of representing the hybrid across models can be described as follows:

[0056]

[0057] Add a shared encoder, and it's ready to use. To rebuild

[0058] Step 4: Loss function and model update.

[0059] like Figure 2 As shown, a queue K = {k1,…,k} is used. N The queue K stores past features for comparison; its length is N. In contrastive learning, all features in queue K are treated as negative samples. o (·) and g m (·) represent the projections of the conventional encoder and the momentum encoder, respectively. Contrast loss L c This can be expressed as:

[0060]

[0061] Where τ is the temperature hyperparameter. o (·) and g m (·) contains global equal pooling and two FC layers respectively.

[0062] After each training iteration, the projection result of the feature map of the (l+1)th layer of the ordinary encoder is... Projection result of the feature map of the (l+1)th layer of the momentum encoder The projection results of the feature maps of the (l+1)th layer of the hybrid encoder The negative samples are pushed into K as further comparisons. j Let j be the j-th past feature stored in the queue. and These represent the feature maps of the (l+1)th layer of the ordinary encoder and the momentum encoder, respectively.

[0063] For different reconstruction contexts, mean squared error (MSE) is used as the default reconstruction loss. Formally, the shared decoder network is represented as... Given that the entire network has a U-Net-like architecture, the input to the decoder should be a multi-layer feature map F. {o,m,h} The calculation of reconstruction loss can be summarized as follows:

[0064]

[0065] Finally, L c and L p The loss function is obtained by merging the parameters proportionally. For the network parameters, the parameters in the ordinary encoder and the momentum encoder are represented as θ. o and θ m θ is updated using the exponential moving average (EMA) factor β. m :

[0066] θ m =βθ m +(1―β)θ o

[0067] It is worth noting that the hybrid encoder has no encoder parameters because it directly obtains a combination of feature maps from the ordinary encoder and the momentum encoder; the hybrid feature map is treated as the input of the shared encoder.

[0068] Example 2

[0069] This invention also discloses a colorectal cancer staging system based on self-supervised learning, used to implement the colorectal cancer staging algorithm based on self-supervised learning as described above. The system includes:

[0070] The first module is set to preserve contrastive representation learning; by reconstructing different contexts, more information is encoded into the representations learned from the contrastive loss.

[0071] The second module is configured to transform the conditional attention mechanism; the transformation vector is encoded into a high-level representation according to the attention mechanism, which forces the encoder to retain information in the learned representation;

[0072] The third module is configured for cross-model blending; the blended features represent image restoration.

[0073] The fourth module is set up for model updates; a fully connected layer and a softmax function are added at the end of the algorithm network, and a loss function is designed to update the model training parameters.

[0074] Finally, it should be noted that the above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A colorectal cancer staging algorithm based on self-supervised learning, characterized in that, The steps include the following: Step 1, Preserve Contrast Representation Learning; By reconstructing different contexts, more information is encoded into the representations learned from the contrastive loss; Step 2, Transform Conditional Attention Mechanism: The transformed vectors are encoded into high-level representations according to the attention mechanism, forcing the encoder to retain information in the learned representations; Step 3, cross-model blending; Cross-model fusion shuffles these feature representations, constructing a new hybrid encoder by mixing feature maps from two different models; therefore, the reconstruction goal of the hybrid encoder is to combine the inputs. ; For network feature maps, use The first part represents the ordinary encoder. Feature mapping of layers, ;same and These are feature maps representing the same location in the momentum encoder and the hybrid encoder, respectively. Add a shared encoder; that is, use To rebuild ; Step 4, Model Update; Add a fully connected layer and... softmax The function involves designing a loss function for model training and parameter updates; a queue is used. Store past features for comparison. The length is In contrastive learning, queues are used. All features are considered negative samples; and These represent the projections of a conventional encoder and a momentum encoder, respectively. and Each layer contains global flat pooling and two fully connected (FC) layers. After each training iteration, the first layer of the ordinary encoder is... Projection results of layer feature maps The momentum encoder's first Projection results of layer feature maps and the hybrid encoder's first Projection results of layer feature maps Push in The negative samples are used for further comparison; for different reconstruction contexts, the mean squared error is used as the default reconstruction loss; formally, the shared decoder network is represented as... The input to the decoder is a multi-layer feature map. Compare the losses and reconstruction loss The loss function is obtained by merging the parameters proportionally; for the network parameters, the parameters in the ordinary encoder and the momentum encoder are expressed as follows: and ; Using the exponential moving average factor renew .

2. The colorectal cancer staging algorithm based on self-supervised learning as described in claim 1, characterized in that, The specific implementation process of the contrastive feature learning described in step 1 is as follows: Preservative contrastive representation learning comprises three distinct encoders and one shared decoder; the encoders and decoders communicate via... U-Net The architecture is connected; first, an exponential moving average is used on the parameters of the ordinary encoder to generate a momentum encoder; then, for each input, the representations of the two encoders are fused across models to construct a hybrid encoder; given a batch of images First, random cropping, random flipping, and random rotation were applied to generate three batches of images for three different encoders. , and Then, low-level processing operations, including inpainting, expainting, and Gaussian blurring, are applied to each batch of images to generate the final input for different encoders. In each training step, including flipping and rotating, three sets of transformations are randomly generated: , and The three sets of transforms are encoded into the last convolutional layer of each encoder; in image reconstruction MSE The true objective of the loss is , and These correspond to different encoders; among them MSE The mean squared error is used to represent the image data. To achieve contrastive learning in the algorithm, past image representations are stored in a queue, and then contrastive loss is applied to positive and negative image pairs.

3. The colorectal cancer staging algorithm based on self-supervised learning as described in claim 1, characterized in that, The specific implementation process of the transition conditional attention mechanism described in step 2 is as follows: For each input, the indicator vector contains a combination of different transformations; each component contains an indicator function, where an indicator function of 1 indicates that a specific transformation is used, and an indicator function of 0 indicates that a specific transformation is not used; the indicator vector is encoded into a high-level feature map using a transformation conditional attention mechanism, which assumes that different channels of the feature map will have different effects on the reconstruction result; in addition, the transformation conditional attention module is only applied to the last convolutional layer of each encoder, i.e., before the FC layer; First, the indicator vector The vector is passed to two fully connected layers (FC) and generated. (A real vector with dimension 1 * number of channels); simultaneously, global average pooling is applied to the high-level feature map of each encoder. (A real vector with dimensions of channel number * depth * height * width) yields a vector ,in Represent the layer index; then calculate and outer product Next, Flatten it and pass it to another fully connected layer to get In order to implement the rescale change, further... Add one sigmoid The function obtains Finally, in and Channel multiplication is used between them, and a convolutional layer with a kernel size of 3 is added.

4. A colorectal cancer staging system based on self-supervised learning, characterized in that, The system is used to implement the colorectal cancer staging algorithm based on self-supervised learning as described in any one of claims 1, the system comprising: The first module is set to preserve contrastive representation learning; by reconstructing different contexts, more information is encoded into the representations learned from the contrastive loss. The second module is configured to transform the conditional attention mechanism; the transformation vector is encoded into a high-level representation according to the attention mechanism, which forces the encoder to retain information in the learned representation; The third module is configured for cross-model blending; the blended features represent image restoration. The fourth module is set up for model updates; a fully connected layer is added at the end of the algorithm network. softmax The function is used to design a loss function for model training and parameter updates.

Images