Anatomy consistency self-supervised learning method and system capable of synthesizing and decomposing medical images

Abstract

The invention provides an anatomy consistency self-supervised learning method and system capable of synthesizing and decomposing a medical image, and the method comprises the steps: S1, generating random image cutting blocks through employing an image cutting strategy based on grid division; s2, sequentially carrying out global consistency learning and local consistency learning according to the image cutting blocks; and S3, integrating learning results of global consistency and local consistency, capturing multi-level information of the medical image by using a total loss function, and optimizing model performance. According to the method, the global and local consistency in the anatomical structure is learned, hierarchical embedded features are extracted from the unlabeled medical image, the ACE framework provides strong representation capability by combining coarse-grained learning of the global features and fine-grained modeling of the local features, and excellent performance can be shown in downstream tasks.

Description

Technical Field

[0001] The present invention relates to the field of medical imaging technology, and in particular to a method and system for self-supervised learning of anatomical consistency of medical images that can be synthesized and decomposed. Background Art

[0002] Medical images play a key role in clinical diagnosis and treatment. Due to the standardized process of acquiring medical images (such as chest X-rays or fundus photography), these images present unique global and local anatomical features. This consistency makes medical images extremely valuable for interpretation. However, the analysis of these images often requires a lot of expertise and traditionally relies on manual annotation to construct training data. However, this annotation process is not only time-consuming and costly, but also affected by the differences between annotators. These challenges have prompted researchers to look for alternatives that do not require manual annotation, among which self-supervised learning (SSL) has gradually become a core method to solve such problems.

[0003] Self-supervised learning (SSL) methods aim to learn meaningful feature representations without labeled data. These methods usually focus on three major themes: learning global features, learning local features, and mining structural patterns and anatomical information from images. Although these studies have made significant progress in the field of general image processing, their limitations are still significant for the anatomical characteristics of medical images.

[0004] An important branch of SSL research focuses on learning global features from images. These methods capture the overall semantic context of the image and ensure the consistency of the input data under different transformations. Methods in this category are generally divided into two categories: contrastive learning methods and non-contrastive learning methods. For example, DINO and BYOL improve model representation in the process of multi-view feature alignment by introducing a contrastive learning framework. However, the core focus of these methods is to capture the consistency of macroscopic structures, so they often perform poorly in obtaining fine-grained local features.

[0005] In contrast, local feature learning aims to extract more refined semantic information by focusing on specific areas of the image. Methodologically, these studies usually rely on detailed pixel-level analysis or image block segmentation to extract more semantically meaningful local features from visual input. Although these methods have shown effectiveness in specific application scenarios, they still face difficulties in dealing with local image block matching. Since the matching method of local embeddings is usually based on semantic similarity or spatial proximity, and lacks an accurate correspondence mechanism, the model may be confused by fuzzy matching.

[0006] In medical images, due to the use of standardized acquisition protocols, the anatomical structure is consistent, which naturally provides a supervisory signal for the self-supervised model. Existing studies have attempted to use this feature to reconstruct the transformed anatomical pattern, analyze the repeatability of anatomical patterns between different patients, or explore the spatial relationship within the anatomical structure. In addition, researchers have also combined adversarial learning technology to further improve the model's performance in anatomical structure consistency.

[0007] Although these methods have made progress in learning anatomical consistency, they ignore the hierarchical relationships in anatomical structures. Anatomical structures usually exist in a hierarchical form. For example, organs can be decomposed into smaller anatomical units, and these units are intrinsically connected. The latest research Adam-v2 attempts to use a hierarchical learning method of combination and decomposition in this direction, but its method is limited to global patterns and ignores the relationship between local image patches.

[0008] Currently, most SSL methods are designed for natural images, such as DINO and BYOL. These methods use contrastive learning or non-contrastive learning to learn global semantic features when the target object background is complex and randomly changing. However, there are essential differences between medical images and photographic images: natural images are often dominated by centered target objects and the background is complex and changing, while medical images, due to the constraints of standardized protocols, show highly consistent anatomical structures. This anatomical consistency is reflected in both macroscopic global features (such as the contours of the lungs and heart) and subtle local features (such as the morphology of the clavicle and bronchi). Figure 1 -a as shown.

[0009] In addition to macroscopic consistency, the anatomical structure of medical images is also characterized by composability and decomposability. For example, the left lung can be decomposed into two independent lobes (upper lobe and lower lobe), and the structures of these two lobes can be combined into a complete left lung, such as Figure 1 -b. These characteristics provide valuable prior knowledge for the design of the model, but existing SSL methods fail to effectively utilize these characteristics. Therefore, fully exploring the anatomical consistency and combination of medical images will provide a new direction for improving the performance of SSL.

[0010] To achieve this goal, we propose a new framework called ACE (Anatomically Consistent Embedding). This framework learns global and local consistency simultaneously through a composition and decomposition strategy. This strategy is inspired by Hinton's research, which shows that humans are able to establish viewpoint-invariant spatial relationships between parts and the whole when parsing visual scenes. ACE uses unlabeled data to learn anatomically consistent embedding features through precise global representation alignment and matrix matching of part-whole relationships. Summary of the invention

[0011] In view of the defects in the prior art, the object of the present invention is to provide a self-supervised learning method and system for anatomical consistency that can be synthesized and decomposed in medical images.

[0012] According to the present invention, a method for self-supervised learning of anatomical consistency of medical images that can be synthesized and decomposed includes:

[0013] Step S1: Generate random image cropping blocks using a grid-based image cropping strategy;

[0014] Step S2: performing global consistency learning and local consistency learning in sequence according to the image cropping blocks;

[0015] Step S3: Integrate the learning results of global consistency and local consistency, use the total loss function to capture the multi-level information of medical images and optimize the model performance.

[0016] Preferably, step S1 includes the following sub-steps:

[0017] Step S1.1: Divide the input image into fixed non-overlapping grids, each grid size is m×m;

[0018] Step S1.2: extract two randomly cropped views C1 and C2 from the input image;

[0019] Step S1.3: In the overlapping area of ​​the cropped views C1 and C2, the grid blocks are further decomposed to obtain sub-blocks of a preset size, thereby completing multi-scale alignment.

[0020] Preferably, the process of global consistency learning includes:

[0021] Extract consistent semantic features from overlapping regions to capture coarse-grained anatomical structures;

[0022] After grid cropping, the two views C1 and C2 are adjusted to the same size:

[0023]

[0024] Among them, C is the number of image channels, H0 and W0 are the height and width of the image respectively;

[0025] Two random data augmentations T1 and T2 are applied to generate enhanced views:

[0026] x = T1(C1) and x ′ =T2(C2);

[0027] Importing augmented views into student models and teacher model In the above example, we generate embedding representations

[0028] y s ,y t ∈R K×N ;

[0029] Among them, K is the dimension of local embedding representation, And (m,m) is the resolution of the sub-block;

[0030] For the overlapping areas of the cropped views, average pooling is used to generate a global embedding representation

[0031]

[0032] Among them, O1 and O2 are the overlapping areas in C1 and C2, which are normalized to the probability distribution P by the softmax function. s and

[0033]

[0034] Among them, τ s It is the temperature parameter used to control the sharpness of the probability distribution;

[0035] Minimize the student distribution P s and teacher distribution P t The cross entropy loss between enables the model to capture global consistency features. The loss function is as follows:

[0036]

[0037] Among them, θ s are the parameters of the student model, and CE(a,b)=-alogb is the cross entropy loss.

[0038] Preferably, the local consistency learning includes local combination and local decomposition to learn fine-grained anatomical features and model the anatomical structure.

[0039] Preferably, the local combination includes combining the embeddings of multiple sub-blocks into a larger overall embedding, encouraging the model to learn the anatomical structure from part to whole; for the overlapping area of ​​the cropped image, the embeddings of multiple sub-blocks are consistent with the embedding of their corresponding whole p, and the combined embedding is defined as:

[0040]

[0041] in, is a feature combiner, is the student model encoder, q1, q2, q3, q4 are sub-blocks; the similarity between matching sub-block embeddings is maximized through the cross-correlation matrix, while minimizing the similarity between mismatching sub-block embeddings; a loss function similar to CLIP is used to guide the model to learn local consistency. When C1 is input to the student model and C2 is input to the teacher model, the resulting embeddings are y s ,y t ∈R K×N , then y s Input to the combiner To combine each 2×2 embedding, the cross-correlation combination matrix is:

[0042]

[0043] in, T is the transpose of the matrix, (·) is the matrix multiplication, and the sigmoid function is used to limit the value range of each element of the matrix to (0,1); optimize the target matrix The element value of is 0 or 1. When it is 1, it means that the two local embeddings match each other in the original position, and when it is 0, they do not match each other. The loss function of the combined embedding is calculated by maximizing the matching embedding similarity and minimizing the mismatching embedding similarity:

[0044]

[0045] Among them, the hyperparameter α is used to control the balance of positive and negative samples.

[0046] Preferably, the local decomposition includes the reverse operation of the local combination process, by decomposing an overall image block into multiple sub-blocks, learning fine-grained anatomical features from the whole to the part; for an overall image block p, through the feature decomposer Decompose it into multiple sub-block embeddings

[0047]

[0048] in, Encoder for the student model;

[0049] Teacher Model The corresponding sub-block embeddings are generated through the corresponding inputs; to ensure the consistency of the anatomical structure, the similarity between the matching sub-block embeddings is maximized through the cross entropy of the correlation matrix, while minimizing the similarity between the mismatched embeddings. The specific loss function is as follows:

[0050]

[0051] Among them, M decompis the matching matrix calculated by local decomposition, T decomp is the target matching matrix, and α is a hyperparameter to balance positive and negative samples.

[0052] Preferably, the total loss function is obtained by integrating the global consistency loss L global and local consistency loss, defined as follows:

[0053]

[0054] Among them, L comp is the combined loss, L decomp To decompose the loss, λ1, λ2, λ3 are weight parameters used to balance different loss terms.

[0055] According to the present invention, a medical image synthesis and decomposition anatomical consistency self-supervised learning system is provided, comprising:

[0056] Module M1: Generate random image cropping blocks using a grid-based image cropping strategy;

[0057] Module M2: Global consistency learning and local consistency learning are performed in sequence according to the image cropping blocks;

[0058] Module M3: Integrate the learning results of global consistency and local consistency, use the total loss function to capture the multi-level information of medical images and optimize the model performance.

[0059] Preferably, the module M1 includes the following submodules:

[0060] Module M1.1: Split the input image into fixed non-overlapping grids, each of which is m×m in size;

[0061] Module M1.2: Extract two random cropped views C1 and C2 from the input image;

[0062] Module M1.3: In the overlapping area of ​​the cropped views C1 and C2, the grid blocks are further decomposed to obtain sub-blocks of preset sizes to complete multi-scale alignment.

[0063] Preferably, the process of global consistency learning includes:

[0064] Extract consistent semantic features from overlapping regions to capture coarse-grained anatomical structures;

[0065] After grid cropping, the two views C1 and C2 are adjusted to the same size:

[0066]

[0067] Among them, C is the number of image channels, H0 and W0 are the height and width of the image respectively;

[0068] Two random data augmentations T1 and T2 are applied to generate enhanced views:

[0069] x = T1(C1) and x ′ =T2(C2);

[0070] Importing augmented views into student models and teacher model In the above example, we generate embedding representations

[0071] y s ,y t ∈R K×K ;

[0072] Among them, K is the dimension of local embedding representation, And (m,m) is the resolution of the sub-block;

[0073] For the overlapping areas of the cropped views, average pooling is used Generating a global embedding representation

[0074]

[0075] Among them, O1 and O2 are the overlapping areas in C1 and C2, which are normalized to the probability distribution P by the softmax function. s and

[0076]

[0077] Among them, τ s It is the temperature parameter used to control the sharpness of the probability distribution;

[0078] Minimize the student distribution P s and teacher distribution P t The cross entropy loss between enables the model to capture global consistency features. The loss function is as follows:

[0079]

[0080] Among them, θ s are the parameters of the student model, and CE(a,b)=-alogb is the cross entropy loss.

[0081] Preferably, the local consistency learning includes local combination and local decomposition to learn fine-grained anatomical features and model the anatomical structure.

[0082] Preferably, the local combination includes combining the embeddings of multiple sub-blocks into a larger overall embedding, encouraging the model to learn the anatomical structure from part to whole; for the overlapping area of ​​the cropped image, the embeddings of multiple sub-blocks are consistent with the embedding of their corresponding whole p, and the combined embedding is defined as:

[0083]

[0084] in, is a feature combiner, is the student model encoder, q1, q2, q3, q4 are sub-blocks; the similarity between matching sub-block embeddings is maximized through the cross-correlation matrix, while minimizing the similarity between mismatching sub-block embeddings; a loss function similar to CLIP is used to guide the model to learn local consistency. When C1 is input to the student model and C2 is input to the teacher model, the resulting embeddings are y s ,y t ∈R K×N , then y s Input to the combiner To combine each 2×2 embedding, the cross-correlation combination matrix is:

[0085]

[0086] Among them, M comp ∈R N×N / 4, T is the transpose of the matrix, (·) is the matrix multiplication, and the sigmoid function is used to limit the value range of each element of the matrix to (0,1); optimize the target matrix T comp ∈R N×N / The element value of 4 is 0 or 1. When it is 1, it means that the two local embeddings match each other in the original position, and when it is 0, they do not match. The loss function of the combined embedding is calculated by maximizing the matching embedding similarity and minimizing the mismatching embedding similarity:

[0087]

[0088] Among them, the hyperparameter α is used to control the balance of positive and negative samples.

[0089] Preferably, the local decomposition includes the reverse operation of the local combination process, by decomposing an overall image block into multiple sub-blocks, learning fine-grained anatomical features from the whole to the part; for an overall image block p, through the feature decomposer Decompose it into multiple sub-block embeddings

[0090]

[0091] in, Encoder for the student model;

[0092] Teacher Model The corresponding sub-block embeddings are generated through the corresponding inputs; to ensure the consistency of the anatomical structure, the similarity between the matching sub-block embeddings is maximized through the cross entropy of the correlation matrix, while minimizing the similarity between the mismatched embeddings. The specific loss function is as follows:

[0093]

[0094] Among them, M decomp is the matching matrix calculated by local decomposition, T decomp is the target matching matrix, and α is a hyperparameter to balance positive and negative samples.

[0095] Preferably, the total loss function is obtained by integrating the global consistency loss L global and local consistency loss, defined as follows:

[0096]

[0097] Among them, L comp is the combined loss, L decomp To decompose the loss, λ1, λ2, λ3 are weight parameters used to balance different loss terms.

[0098] Compared with the prior art, the present invention has the following beneficial effects:

[0099] 1. This paper proposes the ACE framework, a novel SSL method that aims to learn visual representations by combining and decomposing anatomical structures in medical images, and learns global consistency and local consistency through reliable global representation alignment and corresponding matrix matching. It has outstanding characteristics and effective migration capabilities, and shows great potential in promoting and explaining the application of AI in medical image analysis.

[0100] 2. The present invention extracts hierarchical embedded features from unlabeled medical images by learning the global and local consistency in anatomical structures. The ACE framework provides powerful representation capabilities by combining coarse-grained learning of global features with fine-grained modeling of local features, and can demonstrate excellent performance in downstream tasks.

[0101] 3. The global consistency proposed in this invention encourages the network to extract consistent semantic features between different data enhancements from overlapping areas. The local consistency forces the model to learn local fine-grained anatomical patterns through combination and decomposition. By integrating these two components into a unified framework, ACE can capture multi-level information of medical images from coarse to fine, providing effective support for a variety of downstream tasks.

[0102] Other beneficial effects of the present invention will be explained in the specific implementation manner through the introduction of specific technical features and technical solutions. Through the introduction of these technical features and technical solutions, those skilled in the art should be able to understand the beneficial technical effects brought about by the technical features and technical solutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0103] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0104] Figure 1 It is a schematic diagram of the anatomical model in the present invention.

[0105] Figure 2 This is a framework diagram of the method of the present invention.

[0106] Figure 3 Schematic diagram of extracting patches from a test image in the present invention.

[0107] Figure 4 Schematic diagram of extracting a random patch from each image in the present invention.

[0108] Figure 5 A diagram showing the ability of the ACE framework of the present invention to capture semantically rich features in its learned embedding space.

[0109] Figure 6 Schematic diagram of different anatomical landmarks marked in the present invention.

[0110] Figure 7 A schematic diagram of an image used for labeling a query given in the present invention.

[0111] Figure 8 Schematic diagram of the robustness of the ACE framework of the present invention under limited data conditions.

[0112] Fig. 9 The ACE in the present invention provides a generalizable tabular graph for a wide range of downstream tasks.

[0113] Fig.10 Schematic diagram of the seamless extension of the ACE framework of the present invention to other imaging modalities.

[0114] Fig.11 The present invention is a flow chart of the method. DETAILED DESCRIPTION

[0115] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0116] Reference Figure 2 and Fig.11 As shown, a method for self-supervised learning of anatomical consistency of medical images that can be synthesized and decomposed includes:

[0117] Step S1: Generate random image cropping blocks using a grid-based image cropping strategy;

[0118] Step S1.1: Divide the input image into fixed non-overlapping grids, each grid size is m×m;

[0119] Step S1.2: extract two randomly cropped views C1 and C2 from the input image;

[0120] Step S1.3: In the overlapping area of ​​the cropped views C1 and C2, the grid blocks are further decomposed to obtain sub-blocks of a preset size, thereby completing multi-scale alignment.

[0121] Step S2: performing global consistency learning and local consistency learning in sequence according to the image cropping blocks;

[0122] The process of global consistency learning includes:

[0123] Extract consistent semantic features from overlapping regions to capture coarse-grained anatomical structures;

[0124] After grid cropping, the two views C1 and C2 are adjusted to the same size:

[0125]

[0126] Among them, C is the number of image channels, H0 and W0 are the height and width of the image respectively;

[0127] Two random data augmentations T1 and T2 are applied to generate enhanced views:

[0128] x = T1(C1) and x ′ =T2(C2);

[0129] Input the augmented view to the student model f θs and the teacher model f θt In the above example, we generate embedding representations

[0130] y s ,y t ∈RK×N ;

[0131] Among them, K is the dimension of local embedding representation, And (m,m) is the resolution of the sub-block;

[0132] For the overlapping areas of the cropped views, average pooling is used Generating a global embedding representation

[0133]

[0134] Among them, O1 and O2 are the overlapping areas in C1 and C2, which are normalized to the probability distribution P by the softmax function. s and

[0135]

[0136] Among them, τ s It is the temperature parameter used to control the sharpness of the probability distribution;

[0137] Minimize the student distribution P s and teacher distribution P t The cross entropy loss between enables the model to capture global consistency features. The loss function is as follows:

[0138]

[0139] Among them, θ s are the parameters of the student model, and CE(a,b)=-alogb is the cross entropy loss.

[0140] Local consistency learning includes local combination as well as local decomposition to learn fine-grained anatomical features to model anatomical structures.

[0141] Local combination involves combining the embeddings of multiple sub-blocks into a larger overall embedding, encouraging the model to learn the anatomical structure from part to whole; for the overlapping areas of the cropped images, the embeddings of multiple sub-blocks are consistent with the embedding of their corresponding whole p, and the combined embedding is defined as:

[0142]

[0143] in, is a feature combiner, is the student model encoder, q1, q2, q3, q4 are sub-blocks; the similarity between matching sub-block embeddings is maximized through the cross-correlation matrix, while minimizing the similarity between mismatching sub-block embeddings; a loss function similar to CLIP is used to guide the model to learn local consistency. When C1 is input to the student model and C2 is input to the teacher model, the resulting embeddings are y s ,yt ∈R K×N , then y s Input to the combiner C θs (y s )∈R K×N / To combine each 2×2 embedding, the cross-correlation combination matrix is:

[0144]

[0145] Among them, M comp ∈R N×N / 4, T is the transpose of the matrix, (·) is the matrix multiplication, and the sigmoid function is used to limit the value range of each element of the matrix to (0,1); optimize the target matrix T comp ∈R N×N / The element value of 4 is 0 or 1. When it is 1, it means that the two local embeddings match each other in the original position, and when it is 0, they do not match. The loss function of the combined embedding is calculated by maximizing the matching embedding similarity and minimizing the mismatching embedding similarity:

[0146]

[0147] Among them, the hyperparameter α is used to control the balance of positive and negative samples.

[0148] Local decomposition includes the reverse operation of the local combination process, which decomposes an overall image block into multiple sub-blocks to learn fine-grained anatomical features from the whole to the part; for an overall image block p, the feature decomposer Decompose it into multiple sub-block embeddings

[0149]

[0150] in, Encoder for the student model;

[0151] Teacher Model The corresponding sub-block embeddings are generated through the corresponding inputs; to ensure the consistency of the anatomical structure, the similarity between the matching sub-block embeddings is maximized through the cross entropy of the correlation matrix, while minimizing the similarity between the mismatched embeddings. The specific loss function is as follows:

[0152]

[0153] Among them, M decomp is the matching matrix calculated by local decomposition, T decomp is the target matching matrix, and α is a hyperparameter to balance positive and negative samples.

[0154] Step S3: Integrate the learning results of global consistency and local consistency, use the total loss function to capture the multi-level information of medical images and optimize the model performance.

[0155] The total loss function is obtained by integrating the global consistency loss L global and local consistency loss, defined as follows:

[0156]

[0157] Among them, L comp is the combined loss, L decomp To decompose the loss, λ1, λ2, λ3 are weight parameters used to balance different loss terms.

[0158] The present invention extracts hierarchical embedded features from unlabeled medical images by learning the global and local consistency in anatomical structures. The ACE framework provides powerful representation capabilities by combining coarse-grained learning of global features with fine-grained modeling of local features, and can demonstrate excellent performance in downstream tasks.

[0159] The above is a basic embodiment of the present invention. The technical solution of the present invention is further described below through a preferred embodiment.

[0160] Example 1

[0161] A self-supervised learning method for anatomical consistency in medical images that can be synthesized and decomposed, comprising:

[0162] The ACE framework extracts hierarchical embedding features from unlabeled medical images by learning global and local consistency in anatomical structures. By combining coarse-grained learning of global features with fine-grained modeling of local features, ACE provides powerful representation capabilities and can demonstrate excellent performance in downstream tasks. Figure 2 As shown, it contains two core components:

[0163] (1) Global consistency: The network is encouraged to extract consistent semantic features between different data augmentations from overlapping regions.

[0164] (2) Local consistency: By combining and decomposing, the model is forced to learn local fine-grained anatomical patterns. By integrating these two components into a unified framework, ACE is able to capture multi-level information of medical images from coarse to fine, providing effective support for a variety of downstream tasks. The following will introduce the details of image preprocessing, each component, and joint training loss.

[0165] Step 1: Image preprocessing: grid division and image cropping.

[0166] A grid-based image cropping strategy is proposed to generate random image crops.

[0167] The specific steps are as follows:

[0168] (1) The input image is divided into a non-overlapping grid of size G = 32 × 32, and the size of each grid block is m × m (see Figure 2 )

[0169] (2) Extract two randomly cropped views C1 and C2 from the input image, which contain 14×14 and 28×28 sub-grid blocks respectively;

[0170] (3) In the overlapping area of ​​the cropped views C1 and C2, each larger grid block (e.g., p) can be further decomposed into four smaller sub-blocks (q1, q2, q3, q4), thereby achieving multi-scale alignment.

[0171] In the specific implementation, the size of each grid block is 32 × 32, while the resolution of the input image is 1024 × 1024. This strategy ensures accurate matching of local features while providing the model with multi-scale contextual information.

[0172] Step 2: Learn global consistency.

[0173] The goal of learning global consistency is to extract consistent semantic features from overlapping regions to capture coarse-grained anatomical structures. The specific implementation process is as follows: the two views C1 and C2 after grid cropping are adjusted to the same size

[0174]

[0175] Among them, C is the number of image channels, H0, W0 are the image height and width, and two random data enhancements T1 and T2 are applied to generate enhanced views respectively.

[0176] x = T1(C1) and x ′ =T2(C2);

[0177] The enhanced view is imported into the student model and teacher model In the above example, we generate embedding representations

[0178] y s ,y t ∈R K×N ;

[0179] Among them, K is the dimension of local embedding representation, And (m,m) is the resolution of the sub-block; for the overlapping area of ​​the cropped view, average pooling is used Generating a global embedding representation

[0180]

[0181] Among them, O1, O2 are the overlapping areas in C1 and C2, which are then normalized to the probability distribution P by the softmax function. s and P t :

[0182]

[0183] Among them, τ s is the temperature parameter used to control the sharpness of the probability distribution. Using the same method, we can get P t Finally, by minimizing the student distribution P s and teacher distribution P t The cross entropy loss between them, the model can capture the global consistency features, and the loss function is as follows:

[0184]

[0185] Step 3: Learn local consistency.

[0186] The goal of local consistency is to learn fine-grained anatomical features through combination and decomposition. Local consistency is divided into two parts: local combination and local decomposition, which accurately model the anatomical structure from the perspective of part to whole and from the perspective of whole to part, respectively.

[0187] The core idea of ​​local combination is to encourage the model to learn the anatomical structure from parts to the whole by combining the embeddings of multiple sub-blocks into a larger overall embedding. For the overlapping areas of the cropped images, the embeddings of multiple sub-blocks (such as q1, q2, q3, q4) should be consistent with the embedding of their corresponding whole p. Define the combined embedding as:

[0188]

[0189] in, is a feature combiner, is the student model encoder. The goal of the combination is to make the embeddings of these sub-blocks reasonably combined into the embedding of the entire image block. To ensure this consistency, the cross-correlation matrix is ​​used to maximize the similarity between the matching sub-block embeddings and minimize the similarity between the unmatched sub-block embeddings. A loss function similar to CLIP is used to guide the model to learn local consistency.

[0190] Specifically, when C1 is input to the student model and C2 is input to the teacher model, the resulting embeddings are y s ,y t ∈R K×N , then ys Input to the combiner To combine each 2×2 embedding, the cross-correlation combination matrix is:

[0191]

[0192] Among them, M comp ∈R N×N / 4, T is the transpose of the matrix, (·) is the matrix multiplication, and the sigmoid function is used to limit the value range of each element of the matrix to (0,1). Optimize the target matrix T comp ∈R N×N / 4 The element value of is 0 or 1, where 1 means that the two local embeddings match each other in the original position, and 0 means they do not match. The loss function of the combined embedding is calculated by maximizing the matching embedding similarity and minimizing the mismatching embedding similarity:

[0193]

[0194] Local decomposition is the reverse operation of the local combination process. Its goal is to learn fine-grained anatomical features from the whole to the part by decomposing a larger overall image block into multiple sub-blocks. Specifically, for an overall image block p, the feature decomposer Decompose it into multiple sub-block embeddings in Encoder for the student model.

[0195]

[0196] Then the teacher model The corresponding sub-block embeddings are generated through the corresponding inputs. To ensure the consistency of the anatomical structure, the cross entropy of the correlation matrix is ​​used to maximize the similarity between the matching sub-block embeddings and minimize the similarity between the mismatching embeddings. The specific loss function is as follows:

[0197]

[0198] Among them, M decomp is the matching matrix calculated by local decomposition, T decomp is the target matching matrix, and α is a hyperparameter to balance positive and negative samples.

[0199] Step 4: Calculation of the total loss function.

[0200] The total loss function of ACE is obtained by integrating the global consistency loss L global and local consistency loss (combined loss L comp and decomposition loss L decomp ) is defined as follows:

[0201]

[0202] Among them, λ1, λ2, and λ3 are used to balance the weight parameters of different loss terms.

[0203] The pre-training process of ACE uses a two-layer multi-layer perceptron (MLP) structure to integrate local embeddings. In order to effectively learn anatomically consistent feature embeddings, two backbone network architectures were selected to verify the performance of ACE, namely Swin-B (Swin Transformer-B) and ViT-B (Vision Transformer-B). These backbone networks can efficiently process high-resolution medical images and have strong representation capabilities. The ChestX-ray14 dataset was used for pre-training, which contains 110,000 chest X-ray images from different patients, which have been annotated by experts and cover a variety of lung diseases. Unlabeled images with a resolution of 448x448 were used for training, and the training cycle was 100 epochs. To avoid overfitting, standard data augmentation techniques such as random cropping, rotation, color transformation, etc. were used. During the training process, the main task of the ACE framework is to jointly optimize the loss functions of global and local consistency in order to extract anatomical features in medical images. These losses are optimized in a student-teacher framework so that the model can learn the relationship between global and local anatomical structures.

[0204] After pre-training, ACE was fine-tuned using different medical image classification and segmentation tasks to verify its performance in actual tasks. For classification tasks, we fine-tuned lung disease classification on the ChestX-ray14, Shenzhen CXR, and RSNAPneumonia datasets. For these multi-label classification tasks, the AUC (area under the curve) metric was used to evaluate model performance. For segmentation tasks, lung lesion segmentation was fine-tuned on the JSRT, ChestX-Det, and SIIM datasets, and the Dice coefficient was used as the performance evaluation criterion to evaluate the performance of the model in different segmentation tasks. During the fine-tuning process, the pre-trained model of the ACE framework was used to initialize the weights, and the entire network was further trained on the labeled data. We adjust its performance on specific tasks by optimizing the parameters of the entire model.

[0205] Next, we will evaluate the results and present the experimental results of the ACE framework in multiple medical image analysis tasks, including pre-training performance, transfer learning ability, and application effects in classification and segmentation tasks. In order to verify the effectiveness of ACE, a series of experiments were conducted on different datasets and compared with the current state-of-the-art methods.

[0206] (1) Properties learned through pre-training

[0207] ACE enhances feature composability: This example studies ACE’s ability to preserve anatomical composition in its learned embedding space. Figure 3 As shown in Figure 1-a, patches are randomly extracted from the test images of the ChestX-ray14 dataset, and each patch is further decomposed into 2 or 4 non-overlapping sub-patches. Each extracted patch and its sub-patches are resized to a fixed size (i.e., 448×448), and their features are extracted using the ACE pre-trained model as well as other benchmark pre-trained models. Subsequently, the cosine similarity between the embedding of each patch and the average embedding of its sub-patches is calculated, and the similarity distribution is visualized using Gaussian kernel density estimation (KDE). The results are shown in Figure 3 As shown in Figure 2-b, ACE’s distribution not only exhibits a narrower and taller shape than the baseline model, but also the average similarity value between the embeddings of the patch and its components (sub-patches) is shifted towards 1. These observations suggest that ACE effectively integrates compact features while maintaining the compositional integrity of the anatomical structure, echoing its ability to preserve the compositionality of anatomical structures in the learned embedding space.

[0208] ACE enhances the decomposability of features. This example studies the ability of ACE to maintain the decomposability of anatomical structures in its learned embedding space. To this end, the test set of the ChestX-ray14 dataset is first divided into multiple batches, each containing 32 images. In each batch, a random patch is extracted from each image and marked as Figure 4 -C in a j , the size is 30%-60% of the original image. Then, the original image and the image with the removed area (marked as X) are j-excised ) and the extracted patch (removing region C j ) is passed into ACE’s pre-trained model and other benchmark pre-trained models to extract their features. Finally, calculate and The cosine similarity between Is it close The results are as follows Figure 4 -b, ACE significantly outperforms the SSL baseline models. In particular, ACE achieves a high accuracy of 89.01%, compared to 58.88%, 12.71%, 18.90%, 13.07%, 15.38%, and 3.12% achieved by DINO, PEAC, SelfPatch, DropPos, POPAR, and BYOL, respectively. This significant accuracy difference highlights ACE's superior ability to maintain anatomical structure disaggregation in its learned embedding space.

[0209] ACE provides powerful local feature driven global image retrieval. To explore ACE’s ability to capture semantically rich features in its learned embedding space, we first divide the test set of the ChestX-ray14 dataset into multiple batches, each containing 32 images. For each batch, we select a random image X. j , and extract a random patch from it as a query, labeled Figure 5 -C in a. Use ACE's pre-trained model and other benchmark models to extract query patches and the features of each complete image in the batch Then, the cosine similarity between the query patch embedding and all full image embeddings in the batch is calculated. If the highest cosine similarity score corresponds to the query patch and its associated full image, the retrieval is considered correct. Figure 5 -b shows that ACE outperforms other SSL baseline models in retrieval accuracy (94.37%), demonstrating the semantic richness of the representation learned by ACE. This result highlights the potential of ACE in clinical practice, which can accurately identify and retrieve patients with similar pathological findings based on query patches related to specific diseases.

[0210] ACE provides unique anatomical insertion for different structures. Figure 6 As shown in Figure 2, to investigate the ability of ACE to reflect the locality of anatomical structures in its learned embedding space, a dataset of 1,000 images was compiled from the ChestX-ray14 dataset, each of which was annotated with 9 different anatomical landmarks by experts. 2 The original image with a resolution of 448 2 Patches of size are extracted around the location of each landmark point, and then the latent features of each landmark point instance are extracted using ACE's pre-trained model and other baseline pre-trained models (without fine-tuning). These features are visualized through t-SNE plots. Figure 6 As shown in Figure 3, the SSL baseline models—DINO, PEAC, POPAR

[36] , and DropPos—have difficulty generating unique features for different landmarks, resulting in an ambiguity in the embedding space and mixed clusters. However, ACE excels in distinguishing different anatomical landmarks, generating clusters that are well separated in the learned embedding space. This highlights ACE’s ability to develop a rich embedding space in which different anatomical structures have unique representations, while the same anatomical structures across patients have very similar embedding representations.

[0211] ACE provides unsupervised cross-patient anatomical structure correspondence. To demonstrate the effectiveness of ACE in capturing diverse anatomical structures, patch-level features are used to query the same anatomical structures across different patients in a zero-shot setting. Specifically, using the test images of the ChestX-ray14 dataset, 13 anatomical landmarks annotated by experts are selected, such as Figure 7 -a. For a given query image, a 448-bit 2 patches, which come from an initial size of 1024 2 These patches are fed into ACE’s pre-trained backbone network (without fine-tuning) to obtain query features. Then, for other key images, N k patches, the window size is 448 2 , with a step size of 8 (zero padding is used for boundary patches), and then these patches are input into the ACE backbone network to obtain the feature dictionary of the key image Finally, for E q For each query landmark feature in E k The feature with the closest L2 distance in the key image is the prediction result of the corresponding landmark point. Figure 7 As shown, in Figure 7 -a draws the predicted landmarks and true landmarks, and passes Figure 7 The box plot in -b analyzes the prediction error of each landmark point. From the results, the ACE encoded features can accurately detect anatomical landmark points, with an average error of 61 pixels (at 1024 2 The results show that the extracted features reliably represent specific anatomical regions and remain consistent under significant morphological variations.

[0212] (2) Downstream migration ability of ACE.

[0213] Data efficiency evaluation. We analyze the robustness of ACE representations in the presence of limited data. To this end, we compare the pre-trained ACE model with two SSL pre-trained models (POPAR and DINO) by fine-tuning them on the JSRT-Heart dataset

[38] using a small amount of annotated data (2, 5, and 10 samples) and on the SIIM dataset [2] using a limited number of annotations (1%, 10%, and 50%). Figure 8As shown in the figure, ACE outperforms POPAR and DINO models in both cardiac segmentation and pneumothorax classification tasks with limited data. In particular, in the cardiac segmentation task, ACE achieves over 91% of its full training data performance using only 2 labeled samples. These results highlight the annotation efficiency of ACE, especially in target tasks where labeled data is scarce.

[0214] Fine-tuning Evaluation. To investigate the generalization ability of ACE’s representations in a fully fine-tuned setting to various downstream tasks, ACE is compared with 9 SSL baseline models with different objectives, covering 3 classification tasks and 4 segmentation tasks. In addition, the performance of training from scratch is also included as a lower bound.

[0215] like Fig. 9 As shown, the ACE model using the ViT-B backbone network performs competitively or better than baseline models using the same backbone network, including DINO, SelfPatch, and DropPos. In addition, ACE using the Swin-B backbone network consistently outperforms BYOL, DINO, and POPAR in all downstream tasks, even though these models also use the Swin-B backbone network. Compared with adaptive methods for medical imaging, including the visual SSL model POPAR, Adam and the visual-language SSL methods KAD, ChexZero, DeViDe, ACE is able to provide the best or second best performance. These results demonstrate the higher transferability and generalization ability of ACE representations across a variety of tasks.

[0216] ACE outperforms multiple SSL baselines with different objectives and backbones on multiple classification and segmentation tasks in a fully fine-tuned setting. The best method is in bold and the second best method is underlined. In each task, an independent two-sample t-test was performed between the best method (in bold) and the other methods.

[0217] The blue highlighted boxes indicate results with no statistically significant difference at the p = 0.05 level.

[0218] (3) The generalization ability of ACE.

[0219] The ACE framework can be seamlessly extended to other imaging modalities. To demonstrate this, ACE is pre-trained on unlabeled fundus images EyePACS and fine-tuned on the diabetic retinopathy classification dataset EyePACS. Fig.10 As shown in Figure 2-a, ACE outperforms the state-of-the-art SSL method DINO, the large-scale pre-trained method LVM-Med, and the method trained from scratch. In addition, unsupervised anatomical correspondence is performed on the fundus image registration dataset FIRE based on the pre-trained backbone network of ACE without fine-tuning. Fig.10 -b, the key points corresponding to the query image can be accurately located in the key image.

[0220] The present invention aims to learn visual representations by combining and decomposing anatomical structures in medical images, relying on learning global consistency and local consistency through reliable global representation alignment and correspondence matrix matching. ACE has been rigorously tested through comprehensive experiments on various tasks, demonstrating its outstanding properties and effective transfer capabilities, showing great potential in promoting and interpreting AI applications in medical image analysis.

[0221] The present invention also provides a self-supervised learning system for anatomical consistency of medical images that can be synthesized and decomposed. The self-supervised learning system for anatomical consistency of medical images that can be synthesized and decomposed can be realized by executing the process steps of the self-supervised learning method for anatomical consistency of medical images that can be synthesized and decomposed. That is, those skilled in the art can understand the self-supervised learning method for anatomical consistency of medical images that can be synthesized and decomposed as a preferred implementation of the self-supervised learning system for anatomical consistency of medical images that can be synthesized and decomposed.

[0222] Specifically, a medical image synthesis and decomposition anatomical consistency self-supervised learning system includes:

[0223] Module M1: Generate random image cropping blocks using a grid-based image cropping strategy;

[0224] Module M2: Global consistency learning and local consistency learning are performed in sequence according to the image cropping blocks;

[0225] Module M3: Integrate the learning results of global consistency and local consistency, use the total loss function to capture the multi-level information of medical images and optimize the model performance.

[0226] The module M1 includes the following submodules:

[0227] Module M1.1: Split the input image into fixed non-overlapping grids, each of which is m×m in size;

[0228] Module M1.2: Extract two random cropped views C1 and C2 from the input image;

[0229] Module M1.3: In the overlapping area of ​​the cropped views C1 and C2, the grid blocks are further decomposed to obtain sub-blocks of preset sizes to complete multi-scale alignment.

[0230] The process of global consistency learning includes:

[0231] Extract consistent semantic features from overlapping regions to capture coarse-grained anatomical structures;

[0232] After grid cropping, the two views C1 and C2 are adjusted to the same size:

[0233]

[0234] Among them, C is the number of image channels, H0 and W0 are the height and width of the image respectively;

[0235] Two random data augmentations T1 and T2 are applied to generate enhanced views:

[0236] x = T1(C1) and x′ = T2(C2);

[0237] Importing augmented views into student models and teacher model In the above example, we generate embedding representations

[0238] y s ,y t ∈R K×N ;

[0239] Among them, K is the dimension of local embedding representation, And (m,m) is the resolution of the sub-block;

[0240] For the overlapping areas of the cropped views, average pooling is used Generating a global embedding representation

[0241]

[0242] Among them, O1 and O2 are the overlapping areas in C1 and C2, which are normalized to the probability distribution P by the softmax function. s and

[0243]

[0244] Among them, τ s It is the temperature parameter used to control the sharpness of the probability distribution;

[0245] Minimize the student distribution P s and teacher distribution P t The cross entropy loss between enables the model to capture global consistency features. The loss function is as follows:

[0246]

[0247] Among them, θ s are the parameters of the student model, and CE(a,b)=-alogb is the cross entropy loss.

[0248] The local consistency learning includes local combination and local decomposition to learn fine-grained anatomical features to model the anatomical structure.

[0249] The local combination includes combining the embeddings of multiple sub-blocks into a larger overall embedding, encouraging the model to learn the anatomical structure from part to whole; for the overlapping areas of the cropped images, the embeddings of multiple sub-blocks are consistent with the embedding of their corresponding overall p, and the combined embedding is defined as:

[0250]

[0251] in, is a feature combiner, is the student model encoder, q1, q2, q3, q4 are sub-blocks; the similarity between matching sub-block embeddings is maximized through the cross-correlation matrix, while minimizing the similarity between mismatching sub-block embeddings; a loss function similar to CLIP is used to guide the model to learn local consistency. When C1 is input to the student model and C2 is input to the teacher model, the resulting embeddings are y s ,y t ∈R K×N , then y s Input to the combiner To combine each 2×2 embedding, the cross-correlation combination matrix is:

[0252]

[0253] Among them, M comp ∈R N×N / 4 , T is the transpose of the matrix, (·) is the matrix multiplication, and the sigmoid function is used to limit the value range of each element of the matrix to (0,1); optimize the target matrix T comp ∈R N×N / 4 The element value of is 0 or 1. When it is 1, it means that the two local embeddings match each other in the original position, and when it is 0, they do not match each other. The loss function of the combined embedding is calculated by maximizing the matching embedding similarity and minimizing the mismatching embedding similarity:

[0254]

[0255] Among them, the hyperparameter α is used to control the balance of positive and negative samples.

[0256] The local decomposition includes the reverse operation of the local combination process, by decomposing an overall image block into multiple sub-blocks, learning fine-grained anatomical features from the whole to the part; for an overall image block p, through the feature decomposer Decompose it into multiple sub-block embeddings

[0257]

[0258] in, Encoder for the student model;

[0259] Teacher Model The corresponding sub-block embeddings are generated through the corresponding inputs; to ensure the consistency of the anatomical structure, the similarity between the matching sub-block embeddings is maximized through the cross entropy of the correlation matrix, while minimizing the similarity between the mismatched embeddings. The specific loss function is as follows:

[0260]

[0261] Among them, M decomp is the matching matrix calculated by local decomposition, T decomp is the target matching matrix, and α is a hyperparameter to balance positive and negative samples.

[0262] The total loss function is obtained by integrating the global consistency loss L global and local consistency loss, defined as follows:

[0263]

[0264] Among them, L comp is the combined loss, L decomp To decompose the loss, λ1, λ2, λ3 are weight parameters used to balance different loss terms.

[0265] Those skilled in the art know that, in addition to realizing the system and its various devices, modules, and units provided by the present invention in a purely computer-readable program code, it is entirely possible to realize the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.

[0266] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A self-supervised learning method for anatomical consistency of medical images that can be synthesized and decomposed, characterized in that: include: Step S1: Generate random image cropping blocks using a grid-based image cropping strategy; Step S2: performing global consistency learning and local consistency learning in sequence according to the image cropping blocks; Step S3: Integrate the learning results of global consistency and local consistency, use the total loss function to capture the multi-level information of medical images and optimize the model performance.

2. The method for self-supervised learning of anatomical consistency of medical images that can be synthesized and decomposed according to claim 1, characterized in that: The step S1 comprises the following sub-steps: Step S1.1: Divide the input image into fixed non-overlapping grids, each grid has a size of m×m; Step S1.2: extract two randomly cropped views C1 and C2 from the input image; Step S1.3: In the overlapping area of ​​the cropped views C1 and C2, the grid blocks are further decomposed to obtain sub-blocks of a preset size, thereby completing multi-scale alignment.

3. The method for self-supervised learning of anatomical consistency of medical images that can be synthesized and decomposed according to claim 2, characterized in that: The process of global consistency learning includes: Extract consistent semantic features from overlapping regions to capture coarse-grained anatomical structures; After grid cropping, the two views C1 and C2 are adjusted to the same size: Among them, C is the number of image channels, H0 and W0 are the height and width of the image respectively; Two random data augmentations T1 and T2 are applied to generate enhanced views: x = T1(C1) and x ′ =T2(C2); Importing augmented views into student models and teacher model In the above example, we generate embedding representations and s ,and t ∈R K×N ; Among them, K is the dimension of local embedding representation, And (m,m) is the resolution of the sub-block; For the overlapping areas of the cropped views, average pooling is used Generating a global embedding representation Among them, O1 and O2 are the overlapping areas in C1 and C2, which are normalized to the probability distribution P by the softmax function. s and Among them, τ s It is the temperature parameter used to control the sharpness of the probability distribution; Minimize the student distribution P s and teacher distribution P t The cross entropy loss between enables the model to capture global consistency features. The loss function is as follows: Among them, θ s are the parameters of the student model, and CE(a,b)=-alogb is the cross entropy loss.

4. The method for self-supervised learning of anatomical consistency of medical images that can be synthesized and decomposed according to claim 3, characterized in that: The local consistency learning includes local combination and local decomposition to learn fine-grained anatomical features to model the anatomical structure.

5. The method for self-supervised learning of anatomical consistency of medical images that can be synthesized and decomposed according to claim 4, characterized in that: The local combination includes combining the embeddings of multiple sub-blocks into a larger overall embedding, encouraging the model to learn the anatomical structure from part to whole; for the overlapping areas of the cropped images, the embeddings of multiple sub-blocks are consistent with the embedding of their corresponding overall p, and the combined embedding is defined as: in, is a feature combiner, is the student model encoder, q1, q2, q3, q4 are sub-blocks; the similarity between matching sub-block embeddings is maximized through the cross-correlation matrix, while minimizing the similarity between mismatching sub-block embeddings; a loss function similar to CLIP is used to guide the model to learn local consistency. When C1 is input to the student model and C2 is input to the teacher model, the resulting embeddings are y s ,y t ∈R K×N , then y s Input to the combiner To combine each 2×2 embedding, the cross-correlation combination matrix is: Among them, M comp ∈R N×N / 4 , T is the transpose of the matrix, (·) is the matrix multiplication, and the sigmoid function is used to limit the value range of each element of the matrix to (0,1); optimize the target matrix T comp ∈R N×N / 4 The element value of is 0 or 1. When it is 1, it means that the two local embeddings match each other in the original position, and when it is 0, they do not match each other. The loss function of the combined embedding is calculated by maximizing the matching embedding similarity and minimizing the mismatching embedding similarity: Among them, the hyperparameter α is used to control the balance of positive and negative samples.

6. The method for self-supervised learning of anatomical consistency of medical images that can be synthesized and decomposed according to claim 4, characterized in that: The local decomposition includes the reverse operation of the local combination process, by decomposing an overall image block into multiple sub-blocks, learning fine-grained anatomical features from the whole to the part; for an overall image block p, through the feature decomposer Decompose it into multiple sub-block embeddings in, Encoder for the student model; Teacher Model The corresponding sub-block embeddings are generated through the corresponding inputs; to ensure the consistency of the anatomical structure, the similarity between the matching sub-block embeddings is maximized through the cross entropy of the correlation matrix, while minimizing the similarity between the mismatched embeddings. The specific loss function is as follows: Among them, M decomp is the matching matrix calculated by local decomposition, T decomp is the target matching matrix, and α is a hyperparameter to balance positive and negative samples.

7. The method for self-supervised learning of anatomical consistency of medical images that can be synthesized and decomposed according to claim 1, characterized in that: The total loss function is obtained by integrating the global consistency loss L global and local consistency loss, defined as follows: Among them, L comp is the combined loss, L decomp To decompose the loss, λ1, λ2, λ3 are weight parameters used to balance different loss terms.

8. A self-supervised learning system for anatomical consistency that can be synthesized and decomposed in medical images, characterized in that: include: Module M1: Generate random image cropping blocks using a grid-based image cropping strategy; Module M2: Global consistency learning and local consistency learning are performed in sequence according to the image cropping blocks; Module M3: Integrate the learning results of global consistency and local consistency, use the total loss function to capture the multi-level information of medical images and optimize the model performance.

9. The medical image synthesizable and decomposable anatomical consistency self-supervised learning system according to claim 8, characterized in that: The module M1 Includes the following submodules: Module M1.1: Split the input image into fixed non-overlapping grids, each of which is m×m in size; Module M1.2: Extract two random cropped views C1 and C2 from the input image; Module M1.3: In the overlapping area of ​​the cropped views C1 and C2, the grid blocks are further decomposed to obtain sub-blocks of preset sizes to complete multi-scale alignment.

10. The medical image synthesizable and decomposable anatomical consistency self-supervised learning system according to claim 9, characterized in that: The process of global consistency learning includes: Extract consistent semantic features from overlapping regions to capture coarse-grained anatomical structures; After grid cropping, the two views C1 and C2 are adjusted to the same size: Among them, C is the number of image channels, H0 and W0 are the height and width of the image respectively; Two random data augmentations T1 and T2 are applied to generate enhanced views: x = T1(C1) and x ′ =T2(C2); Importing augmented views into student models and teacher model In the above example, we generate embedding representations and s ,and t ∈R K×N ; Among them, K is the dimension of local embedding representation, And (m,m) is the resolution of the sub-block; For the overlapping areas of the cropped views, average pooling is used Generating a global embedding representation Among them, O1 and O2 are the overlapping areas in C1 and C2, which are normalized to the probability distribution P by the softmax function. s and Among them, τ s It is the temperature parameter used to control the sharpness of the probability distribution; Minimize the student distribution P s and teacher distribution P t The cross entropy loss between enables the model to capture global consistency features. The loss function is as follows: Among them, θ s are the parameters of the student model, and CE(a,b)=-alogb is the cross entropy loss.

Images