A Self-Supervised Learning Method and System for Anatomical Consistency Synthesization and Decomposition of Medical Images

Abstract

This invention provides a self-supervised learning method and system for anatomical consistency in medical images that can be synthesized and decomposed. The method includes steps S1: generating random image cropping blocks using a grid-based image cropping strategy; step S2: performing global consistency learning and local consistency learning sequentially based on the image cropping blocks; and step S3: integrating the learning results of global and local consistency, using a total loss function to capture multi-level information of the medical image and optimize model performance. This invention extracts hierarchical embedding features from unlabeled medical images by learning global and local consistency in anatomical structures. The ACE framework, by combining coarse-grained learning of global features with fine-grained modeling of local features, provides powerful representation capabilities and demonstrates excellent performance in downstream tasks.

Description

Technical Field

[0001] This invention relates to the field of medical imaging technology, and more specifically, to a self-supervised learning method and system for anatomical consistency of medical images that can be synthesized and decomposed. Background Technology

[0002] Medical imaging plays a crucial role in clinical diagnosis and treatment. Due to standardized procedures for acquiring medical images (e.g., chest X-rays or fundus photography), these images exhibit unique global and local anatomical features. This consistency gives medical images extremely high interpretive value. However, the analysis of these images often requires extensive expertise and has traditionally relied on manual annotation to construct training data. This annotation process is not only time-consuming and costly but also susceptible to variations among annotators. These challenges have prompted researchers to seek alternatives that eliminate the need for manual annotation, with self-supervised learning (SSL) gradually emerging as a core method for addressing such problems.

[0003] Self-supervised learning (SSL) methods aim to learn meaningful feature representations without the need for labeled data. These methods typically revolve around three main themes: learning global features, learning local features, and extracting structural patterns and anatomical information from images. Although these studies have made significant progress in general image processing, their limitations remain significant when dealing with the anatomical characteristics of medical images.

[0004] An important branch of SSL research focuses on learning global features from images. These methods ensure the consistency of input data under different transformations by capturing the overall semantic context of the image. This category of methods is generally divided into two main types: contrastive learning methods and non-contrastive learning methods. For example, DINO and BYOL improve model representation during multi-view feature alignment by introducing a contrastive learning framework. However, these methods are primarily concerned with capturing the consistency of macroscopic structures, and therefore often perform poorly in acquiring fine-grained local features.

[0005] In contrast, local feature learning aims to extract more refined semantic information by focusing on specific regions of an image. Methodologically, these studies typically rely on pixel-level detailed analysis or image patch segmentation to extract more semantically meaningful local features from visual input. While these methods have shown effectiveness in specific applications, they still face challenges in handling local image patch matching. Because local embedding matching is usually based on semantic similarity or spatial proximity, lacking a precise correspondence mechanism, the model may become confused by fuzzy matching.

[0006] In medical images, the standardized acquisition protocols ensure consistency in anatomical structures, naturally providing supervisory signals for self-supervised models. Existing research has attempted to leverage this characteristic by reconstructing transformed anatomical patterns to analyze the reproducibility of anatomical patterns among different patients or to explore spatial relationships within anatomical structures. Furthermore, researchers have incorporated adversarial learning techniques to further enhance the model's performance in anatomical consistency.

[0007] While these methods have made progress in learning anatomical consistency, they neglect the hierarchical relationships that exist within anatomical structures. Anatomical structures often exist in a hierarchical form; for example, organs can be broken down into smaller anatomical units, which are intrinsically connected. The recent study Adam-v2 attempts to utilize a hierarchical learning approach of combination and decomposition in this direction, but its method is limited to global patterns and ignores the relationships between local image patches.

[0008] Currently, most SSL methods are designed for natural images, such as DINO and BYOL. These methods utilize contrastive or non-contrastive learning to learn global semantic features even when the background of the target object is complex and randomly varying. However, medical images differ fundamentally from photographic images: natural images often feature a centered target object against a complex and varied background, while medical images, constrained by standardized protocols, exhibit highly consistent anatomical structures. This anatomical consistency is reflected both in macroscopic global features (e.g., the outlines of the lungs and heart) and in subtle local features (e.g., the morphology of the clavicle and bronchi), such as... Figure 1 -a is shown.

[0009] Besides macroscopic consistency, the anatomical structures in medical images also possess the characteristics of composability and decomposability. For example, the left lung can be decomposed into two independent lobes (upper and lower lobes), and the structures of these two lobes can be combined to form a complete left lung, such as... Figure 1 As shown in -b. These characteristics provide valuable prior knowledge for model design, but existing SSL methods have failed to effectively utilize these properties. Therefore, fully exploring the anatomical consistency and composability of medical images will provide a new direction for improving SSL performance.

[0010] To achieve this goal, a novel framework called ACE (Anatomically Consistent Embedding) is proposed. This framework learns global and local consistency simultaneously through a combination and decomposition strategy. This strategy is inspired by Hinton's research, which shows that humans can establish viewpoint-invariant spatial relationships between parts and the whole when resolving visual scenes. ACE utilizes unlabeled data to learn anatomically consistent embedding features through precise global representation alignment and matrix matching of part-whole relationships. Summary of the Invention

[0011] To address the shortcomings of existing technologies, the purpose of this invention is to provide a self-supervised learning method and system for anatomical consistency in medical image synthesis and decomposition.

[0012] A self-supervised learning method for anatomical consistency of medical images that can be synthesized and decomposed according to the present invention includes:

[0013] Step S1: Generate random image cropping blocks using a grid-based image cropping strategy;

[0014] Step S2: Perform global consistency learning and local consistency learning sequentially based on the image cropping blocks;

[0015] Step S3: Integrate the learning results of global consistency and local consistency, use the total loss function to capture multi-level information of medical images and optimize model performance.

[0016] Preferably, step S1 includes the following sub-steps:

[0017] Step S1.1: Divide the input image into a fixed non-overlapping grid, with each grid having a size of m×m;

[0018] Step S1.2: Extract two randomly cropped views C1 and C2 from the input image;

[0019] Step S1.3: In the overlapping area of ​​cropped views C1 and C2, further decompose the grid blocks to obtain sub-blocks of preset size, and complete multi-scale alignment.

[0020] Preferably, the global consistency learning process includes:

[0021] Extract consistent semantic features from overlapping regions to capture coarse-grained anatomical structures;

[0022] After being clipped by the grid, the two views C1 and C2 are adjusted to the same size:

[0023]

[0024] Where C is the number of image channels, and H0 and W0 are the height and width of the image, respectively;

[0025] Two random data augmentations, T1 and T2, are applied respectively to generate augmented views:

[0026] x = T1(C1) and x ′ =T2(C2);

[0027] Input the augmented view into the student model Teacher Model In the middle, respectively generate embedded representations

[0028] y s ,y t ∈R K×N ;

[0029] Where K is the dimension of the local embedding representation. And (m,m) is the resolution of the sub-block;

[0030] For the overlapping regions of the clipped view, average pooling is used to generate a global embedding representation.

[0031]

[0032] Where O1 and O2 are the overlapping regions in C1 and C2, which are normalized to a probability distribution P by the softmax function. s and

[0033]

[0034] Where, τ s It is a temperature parameter used to control the sharpness of the probability distribution;

[0035] Minimize student distribution P s Teacher distribution P t The cross-entropy loss between the two features allows the model to capture globally consistent features. The loss function is as follows:

[0036]

[0037] Where, θ s These are the parameters of the student model, and CE(a,b) = -alogb is the cross-entropy loss.

[0038] Preferably, the local consistency learning includes local combination and local decomposition to learn fine-grained anatomical features and model anatomical structures.

[0039] Preferably, the local combination includes combining the embeddings of multiple sub-blocks into a larger overall embedding, encouraging the model to learn the anatomical structure from part to whole; for overlapping regions of the cropped image, the embeddings of multiple sub-blocks are consistent with the embedding of their corresponding overall p, and the combined embedding is defined as:

[0040]

[0041] in, It is a feature combiner. This is the student model encoder, where q1, q2, q3, and q4 are sub-blocks. The similarity between matching sub-block embeddings is maximized using the cross-correlation matrix, while the similarity between mismatched sub-block embeddings is minimized. A loss function similar to CLIP is used to guide the model in learning local consistency. When C1 is input into the student model and C2 is input into the teacher model, the resulting embeddings are y... s ,y t ∈R K×N Then y s Input to combiner To combine each 2×2 embedding, the cross-correlation combination matrix is:

[0042]

[0043] in, T is the transpose of the matrix, (·) represents matrix multiplication, and the sigmoid function is used to restrict the value of each element of the matrix to (0,1); optimize the target matrix. The element value is either 0 or 1, where 1 indicates that the two local embeddings match at their original positions, and 0 indicates that they do not match. The loss function for combined embeddings is calculated by maximizing the similarity of matching embeddings and minimizing the similarity of mismatched embeddings.

[0044]

[0045] The hyperparameter α is used to control the balance between positive and negative samples.

[0046] Preferably, the local decomposition includes the inverse operation of the local combination process, learning fine-grained anatomical features from the whole to the parts by decomposing a whole image patch into multiple sub-patterns; for a whole image patch p, through a feature decomposer Decompose it into the embedding of multiple sub-blocks

[0047]

[0048] in, The encoder for the student model;

[0049] Teacher Model Embeddings of corresponding sub-blocks are generated based on the corresponding inputs. To ensure the consistency of the anatomical structure, the similarity between matching sub-block embeddings is maximized by the cross-entropy of the cross-correlation matrix, while the similarity between mismatched embeddings is minimized. The specific loss function is as follows:

[0050]

[0051] Among them, M decompIt is the matching matrix calculated by local decomposition, T decomp α is the target matching matrix, and α is the hyperparameter for balancing positive and negative samples.

[0052] Preferably, the total loss function integrates the global consistency loss L global The local consistency loss is defined as follows:

[0053]

[0054] Among them, L comp For the combination loss, L decomp To decompose the loss, λ1, λ2, and λ3 are weighting parameters used to balance the different loss terms.

[0055] A self-supervised learning system for anatomical consistency of medical images that can be synthesized and decomposed according to the present invention includes:

[0056] Module M1: Generates random image cropping blocks using a grid-based image cropping strategy;

[0057] Module M2: Performs global consistency learning and local consistency learning sequentially based on the image cropping blocks;

[0058] Module M3: Integrates the learning results of global consistency and local consistency, uses the total loss function to capture multi-level information of medical images and optimize model performance.

[0059] Preferably, module M1 includes the following sub-modules:

[0060] Module M1.1: Divides the input image into a fixed non-overlapping grid, with each grid measuring m×m;

[0061] Module M1.2: Extracts two randomly cropped views, C1 and C2, from the input image;

[0062] Module M1.3: In the overlapping area of ​​clipping views C1 and C2, further decompose the mesh blocks to obtain sub-blocks of preset size, and complete multi-scale alignment.

[0063] Preferably, the global consistency learning process includes:

[0064] Extract consistent semantic features from overlapping regions to capture coarse-grained anatomical structures;

[0065] After being clipped by the grid, the two views C1 and C2 are adjusted to the same size:

[0066]

[0067] Where C is the number of image channels, and H0 and W0 are the height and width of the image, respectively;

[0068] Two random data augmentations, T1 and T2, are applied respectively to generate augmented views:

[0069] x = T1(C1) and x ′ =T2(C2);

[0070] Input the augmented view into the student model Teacher Model In the middle, respectively generate embedded representations

[0071] y s ,y t ∈R K×K ;

[0072] Where K is the dimension of the local embedding representation. And (m,m) is the resolution of the sub-block;

[0073] For the overlapping areas of the clipped view, average pooling is used. Generate global embedding representation

[0074]

[0075] Where O1 and O2 are the overlapping regions in C1 and C2, which are normalized to a probability distribution P by the softmax function. s and

[0076]

[0077] Where, τ s It is a temperature parameter used to control the sharpness of the probability distribution;

[0078] Minimize student distribution P s Teacher distribution P t The cross-entropy loss between the two features allows the model to capture globally consistent features. The loss function is as follows:

[0079]

[0080] Where, θ s These are the parameters of the student model, and CE(a,b) = -alogb is the cross-entropy loss.

[0081] Preferably, the local consistency learning includes local combination and local decomposition to learn fine-grained anatomical features and model anatomical structures.

[0082] Preferably, the local combination includes combining the embeddings of multiple sub-blocks into a larger overall embedding, encouraging the model to learn the anatomical structure from part to whole; for overlapping regions of the cropped image, the embeddings of multiple sub-blocks are consistent with the embedding of their corresponding overall p, and the combined embedding is defined as:

[0083]

[0084] in, It is a feature combiner. This is the student model encoder, where q1, q2, q3, and q4 are sub-blocks. The similarity between matching sub-block embeddings is maximized using the cross-correlation matrix, while the similarity between mismatched sub-block embeddings is minimized. A loss function similar to CLIP is used to guide the model in learning local consistency. When C1 is input into the student model and C2 is input into the teacher model, the resulting embeddings are y... s ,y t ∈R K×N Then y s Input to combiner To combine each 2×2 embedding, the cross-correlation combination matrix is:

[0085]

[0086] Among them, M comp ∈R N×N / 4, T is the transpose of the matrix, (·) is matrix multiplication, and the sigmoid function is used to restrict the value of each element of the matrix to (0,1); optimize the target matrix T comp ∈R N×N / The element value of 4 is either 0 or 1. A value of 1 indicates that the two local embeddings match at their original positions, while a value of 0 indicates that they do not match. The loss function for combined embeddings is calculated by maximizing the similarity of matching embeddings and minimizing the similarity of mismatched embeddings.

[0087]

[0088] The hyperparameter α is used to control the balance between positive and negative samples.

[0089] Preferably, the local decomposition includes the inverse operation of the local combination process, learning fine-grained anatomical features from the whole to the parts by decomposing a whole image patch into multiple sub-patterns; for a whole image patch p, through a feature decomposer Decompose it into the embedding of multiple sub-blocks

[0090]

[0091] in, The encoder for the student model;

[0092] Teacher Model Embeddings of corresponding sub-blocks are generated based on the corresponding inputs. To ensure the consistency of the anatomical structure, the similarity between matching sub-block embeddings is maximized by the cross-entropy of the cross-correlation matrix, while the similarity between mismatched embeddings is minimized. The specific loss function is as follows:

[0093]

[0094] Among them, M decomp It is the matching matrix calculated by local decomposition, T decomp α is the target matching matrix, and α is the hyperparameter for balancing positive and negative samples.

[0095] Preferably, the total loss function integrates the global consistency loss L global The local consistency loss is defined as follows:

[0096]

[0097] Among them, L comp For the combination loss, L decomp To decompose the loss, λ1, λ2, and λ3 are weighting parameters used to balance the different loss terms.

[0098] Compared with the prior art, the present invention has the following beneficial effects:

[0099] 1. This invention proposes the ACE framework, a novel SSL method, which aims to learn visual representations by combining and decomposing anatomical structures in medical images. It learns global and local consistency through reliable global representation alignment and correspondence matrix matching, exhibiting outstanding characteristics and effective transferability, demonstrating great potential in promoting and interpretable AI applications in medical image analysis.

[0100] 2. This invention extracts hierarchical embedding features from unlabeled medical images by learning the global and local consistency in anatomical structures. The ACE framework provides powerful representation capabilities by combining coarse-grained learning of global features with fine-grained modeling of local features, and can demonstrate excellent performance in downstream tasks.

[0101] 3. The global consistency proposed in this invention encourages the network to extract consistent semantic features between different data augmentations from overlapping regions. Local consistency forces the model to learn local fine-grained anatomical patterns through combination and decomposition. By integrating these two components into a unified framework, ACE can capture multi-level information of medical images from coarse to fine, providing effective support for a variety of downstream tasks.

[0102] Other beneficial effects of the present invention will be explained in detail through the introduction of specific technical features and technical solutions in specific embodiments. Those skilled in the art should be able to understand the beneficial technical effects brought about by these technical features and technical solutions through the introduction of these technical features and technical solutions. Attached Figure Description

[0103] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0104] Figure 1 This is a schematic diagram of the anatomical model in this invention.

[0105] Figure 2 This is a schematic diagram of the method framework of the present invention.

[0106] Figure 3 This is a schematic diagram illustrating the extraction of patches from a test image in this invention.

[0107] Figure 4 This is a schematic diagram illustrating the extraction of a random patch from each image in this invention.

[0108] Figure 5 This diagram illustrates the ability of the ACE framework of the present invention to capture semantically rich features in its learned embedding space.

[0109] Figure 6 This is a schematic diagram of the different anatomical landmarks marked in this invention.

[0110] Figure 7 This is a schematic diagram of an image used for tagging queries, as provided in this invention.

[0111] Figure 8 This is a schematic diagram illustrating the robustness of the ACE framework of the present invention under limited data conditions.

[0112] Figure 9 The ACE in this invention provides a generalizable tabular diagram for a wide range of downstream tasks.

[0113] Figure 10 This is a schematic diagram illustrating how the ACE framework of the present invention can be seamlessly extended to other imaging modalities.

[0114] Figure 11 This is a flowchart of the method of the present invention. Detailed Implementation

[0115] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0116] Reference Figure 2 and Figure 11 As shown, a self-supervised learning method for anatomical consistency in medical image synthesis and decomposition includes:

[0117] Step S1: Generate random image cropping blocks using a grid-based image cropping strategy;

[0118] Step S1.1: Divide the input image into a fixed non-overlapping grid, with each grid having a size of m×m;

[0119] Step S1.2: Extract two randomly cropped views C1 and C2 from the input image;

[0120] Step S1.3: In the overlapping area of ​​cropped views C1 and C2, further decompose the grid blocks to obtain sub-blocks of preset size, and complete multi-scale alignment.

[0121] Step S2: Perform global consistency learning and local consistency learning sequentially based on the image cropping blocks;

[0122] The process of global consistency learning includes:

[0123] Extract consistent semantic features from overlapping regions to capture coarse-grained anatomical structures;

[0124] After being clipped by the grid, the two views C1 and C2 are adjusted to the same size:

[0125]

[0126] Where C is the number of image channels, and H0 and W0 are the height and width of the image, respectively;

[0127] Two random data augmentations, T1 and T2, are applied respectively to generate augmented views:

[0128] x = T1(C1) and x ′ =T2(C2);

[0129] Input the augmented view into the student model f θs And teacher model f θt In the middle, respectively generate embedded representations

[0130] y s ,y t ∈RK×N ;

[0131] Where K is the dimension of the local embedding representation. And (m,m) is the resolution of the sub-block;

[0132] For the overlapping areas of the clipped view, average pooling is used. Generate global embedding representation

[0133]

[0134] Where O1 and O2 are the overlapping regions in C1 and C2, which are normalized to a probability distribution P by the softmax function. s and

[0135]

[0136] Where, τ s It is a temperature parameter used to control the sharpness of the probability distribution;

[0137] Minimize student distribution P s Teacher distribution P t The cross-entropy loss between the two features allows the model to capture globally consistent features. The loss function is as follows:

[0138]

[0139] Where, θ s These are the parameters of the student model, and CE(a,b) = -alogb is the cross-entropy loss.

[0140] Local consistency learning includes local combination and local decomposition to learn fine-grained anatomical features and model anatomical structures.

[0141] Local compositing involves combining the embeddings of multiple sub-blocks into a larger overall embedding, encouraging the model to learn the anatomical structure from parts to the whole. For overlapping regions of the cropped image, the embeddings of multiple sub-blocks are consistent with the embedding of their corresponding overall p, and the combined embedding is defined as:

[0142]

[0143] in, It is a feature combiner. This is the student model encoder, where q1, q2, q3, and q4 are sub-blocks. The similarity between matching sub-block embeddings is maximized using the cross-correlation matrix, while the similarity between mismatched sub-block embeddings is minimized. A loss function similar to CLIP is used to guide the model in learning local consistency. When C1 is input into the student model and C2 is input into the teacher model, the resulting embeddings are y... s ,yt ∈R K×N Then y s Input to combiner C θs (y s )∈R K×N / To combine each 2×2 embedding, the cross-correlation combination matrix is:

[0144]

[0145] Among them, M comp ∈R N×N / 4, T is the transpose of the matrix, (·) is matrix multiplication, and the sigmoid function is used to restrict the value of each element of the matrix to (0,1); optimize the target matrix T comp ∈R N×N / The element value of 4 is either 0 or 1. A value of 1 indicates that the two local embeddings match at their original positions, while a value of 0 indicates that they do not match. The loss function for combined embeddings is calculated by maximizing the similarity of matching embeddings and minimizing the similarity of mismatched embeddings.

[0146]

[0147] The hyperparameter α is used to control the balance between positive and negative samples.

[0148] Local decomposition involves the inverse operation of the local combination process, learning fine-grained anatomical features from the whole to the parts by decomposing a whole image patch into multiple sub-patterns; for a whole image patch p, through a feature decomposer... Decompose it into the embedding of multiple sub-blocks

[0149]

[0150] in, The encoder for the student model;

[0151] Teacher Model Embeddings of corresponding sub-blocks are generated based on the corresponding inputs. To ensure the consistency of the anatomical structure, the similarity between matching sub-block embeddings is maximized by the cross-entropy of the cross-correlation matrix, while the similarity between mismatched embeddings is minimized. The specific loss function is as follows:

[0152]

[0153] Among them, M decomp It is the matching matrix calculated by local decomposition, T decomp α is the target matching matrix, and α is the hyperparameter for balancing positive and negative samples.

[0154] Step S3: Integrate the learning results of global consistency and local consistency, use the total loss function to capture multi-level information of medical images and optimize model performance.

[0155] The total loss function incorporates the global consistency loss L global The local consistency loss is defined as follows:

[0156]

[0157] Among them, L comp For the combination loss, L decomp To decompose the loss, λ1, λ2, and λ3 are weighting parameters used to balance the different loss terms.

[0158] This invention extracts hierarchical embedding features from unlabeled medical images by learning the global and local consistency of anatomical structures. The ACE framework provides powerful representation capabilities by combining coarse-grained learning of global features with fine-grained modeling of local features, and can demonstrate excellent performance in downstream tasks.

[0159] The above are basic embodiments of the present invention. The technical solution of the present invention will be further described below through a preferred embodiment.

[0160] Example 1

[0161] A self-supervised learning method for anatomical consistency of medical images that can be synthesized and decomposed includes:

[0162] The ACE framework extracts hierarchical embedding features from unannotated medical images by learning global and local consistency in anatomical structures. By combining coarse-grained learning of global features with fine-grained modeling of local features, ACE provides powerful representation capabilities and demonstrates excellent performance in downstream tasks. The framework is as follows: Figure 2 As shown, it contains two core components:

[0163] (1) Global consistency: Encourage the network to extract consistent semantic features between different data augmentations from overlapping regions.

[0164] (2) Local Consistency: By combining and decomposing elements, the model is forced to learn fine-grained local anatomical patterns. By integrating these two components into a unified framework, ACE can capture multi-level information from medical images from coarse to fine, providing effective support for various downstream tasks. The details of image preprocessing, each component, and joint training loss will be introduced below.

[0165] Step 1: Image preprocessing: grid generation and image cropping.

[0166] A grid-based image cropping strategy is proposed to generate random image cropping blocks.

[0167] The specific steps are as follows:

[0168] (1) The input image is segmented into a non-overlapping grid of size G = 32 × 32, and the size of each grid block is m × m (see...). Figure 2 (The white grid in the middle);

[0169] (2) Extract two randomly cropped views C1 and C2 from the input image, which contain 14×14 and 28×28 sub-grid blocks respectively;

[0170] (3) In the overlapping area of ​​clipping views C1 and C2, each larger grid block (such as p) can be further decomposed into four smaller sub-blocks (q1, q2, q3, q4) to achieve multi-scale alignment.

[0171] In the specific implementation, each grid block is 32×32 pixels, while the input image resolution is 1024×1024. This strategy ensures accurate matching of local features while providing the model with multi-scale contextual information.

[0172] Step 2: Learn global consistency.

[0173] The goal of learning global consistency is to extract consistent semantic features from overlapping regions to capture coarse-grained anatomical structures. The specific implementation process is as follows: Two views, C1 and C2, after being clipped by a mesh, are resized to the same size.

[0174]

[0175] Where C is the number of image channels, H0 and W0 are the image height and width, and two random data augmentation methods T1 and T2 are applied to generate augmented views respectively.

[0176] x = T1(C1) and x ′ =T2(C2);

[0177] The augmented view is input into the student model. Teacher Model In the middle, respectively generate embedded representations

[0178] y s ,y t ∈R K×N ;

[0179] Where K is the dimension of the local embedding representation. And (m,m) represents the resolution of the sub-block; for the overlapping areas of the clipped view, average pooling is used. Generate global embedding representation

[0180]

[0181] Where O1 and O2 are the overlapping regions in C1 and C2, and are then normalized to a probability distribution P using the softmax function. s and P t :

[0182]

[0183] Where, τ s It is a temperature parameter used to control the sharpness of the probability distribution. We can obtain P using the same method. t Finally, by minimizing the student distribution P... s Teacher distribution P t The model captures globally consistent features by using cross-entropy loss, and the loss function is as follows:

[0184]

[0185] Step 3: Learn local consistency.

[0186] The goal of local consistency is to learn fine-grained anatomical features through combination and decomposition. Local consistency consists of two parts: local combination and local decomposition, which accurately model anatomical structures from the perspectives of part-to-whole and whole-to-part, respectively.

[0187] The core idea of ​​local ensemble embedding is to encourage the model to learn the anatomical structure from parts to the whole by combining the embeddings of multiple sub-blocks into a larger overall embedding. For overlapping regions of a cropped image, the embeddings of multiple sub-blocks (e.g., q1, q2, q3, q4) should be consistent with the embedding of their corresponding overall p. The ensemble embedding is defined as:

[0188]

[0189] in, It is a feature combiner. This is the student model encoder. The goal of the combination is to ensure that the embeddings of these sub-blocks can be reasonably combined into the embedding of the entire image patch. To ensure this consistency, the similarity between matching sub-block embeddings is maximized using a cross-correlation matrix, while the similarity between mismatched sub-block embeddings is minimized. A loss function similar to CLIP is used to guide the model in learning local consistency.

[0190] Specifically, when C1 is input into the student model and C2 is input into the teacher model, the resulting embeddings are y and y respectively. s ,y t ∈R K×N , then ys Input to combiner To combine each 2×2 embedding, the cross-correlation combination matrix is:

[0191]

[0192] Among them, M comp ∈R N×N / 4. T is the transpose of the matrix, (·) represents matrix multiplication, and the sigmoid function is used to restrict the value of each element of the matrix to (0,1). Optimize the objective matrix T. comp ∈R N×N / 4 The element values ​​are either 0 or 1, where 1 indicates that the two local embeddings match at their original positions, and 0 indicates that they do not match. The loss function for combined embeddings is calculated by maximizing the similarity of matching embeddings and minimizing the similarity of mismatched embeddings:

[0193]

[0194] Local decomposition is the inverse operation of the local composition process. Its goal is to learn fine-grained anatomical features from the whole to the parts by decomposing a large overall image patch into multiple sub-patterns. Specifically, for an overall image patch p, a feature decomposer... Decompose it into the embedding of multiple sub-blocks in The encoder for the student model.

[0195]

[0196] Then the teacher model The system generates embeddings for corresponding sub-blocks based on the input. To ensure consistency in the anatomical structure, the cross-entropy of the cross-correlation matrix is ​​used to maximize the similarity between matching sub-block embeddings while minimizing the similarity between mismatched embeddings. The specific loss function is as follows:

[0197]

[0198] Among them, M decomp It is the matching matrix calculated by local decomposition, T decomp α is the target matching matrix, and α is the hyperparameter for balancing positive and negative samples.

[0199] Step 4: Calculate the total loss function.

[0200] The total loss function of ACE incorporates the global consistency loss L global and local consistency loss (combined loss L) comp With decomposition loss L decomp The definition is as follows:

[0201]

[0202] λ1, λ2, and λ3 are used as weight parameters to balance different loss terms.

[0203] The pre-training process of ACE utilizes a two-layer multilayer perceptron (MLP) architecture to integrate local embeddings. To effectively learn anatomically consistent feature embeddings, two backbone network architectures were chosen to validate ACE's performance: Swin-B (Swin Transformer-B) and ViT-B (Vision Transformer-B). These backbone networks are capable of efficiently processing high-resolution medical images and possess strong representational power. Pre-training was performed using the ChestX-ray14 dataset, which contains 110,000 chest X-ray images from various patients, expert-annotated and covering a variety of lung diseases. Training was conducted using unannotated 448x448 resolution images for 100 epochs. To avoid overfitting, standard data augmentation techniques such as random cropping, rotation, and color transformation were employed. During training, the primary task of the ACE framework is to jointly optimize global and local consistency loss functions to extract anatomical features from medical images. These losses are optimized within a student-teacher framework, enabling the model to learn the relationships between global and local anatomical structures.

[0204] After pre-training, ACE was fine-tuned using different medical image classification and segmentation tasks to validate its performance on real-world tasks. For classification tasks, we fine-tuned the model for lung disease classification on the ChestX-ray14, Shenzhen CXR, and RSNAPneumonia datasets. The AUC (Area Under the Curve) metric was used to evaluate model performance for these multi-label classification tasks. For segmentation tasks, we fine-tuned the model for lung lesion segmentation on the JSRT, ChestX-Det, and SIIM datasets, using the Dice coefficient as the performance evaluation metric to assess the model's performance on different segmentation tasks. During fine-tuning, the pre-trained model of the ACE framework was used to initialize the weights, and the entire network was further trained on labeled data. We tuned the model's performance on specific tasks by optimizing the parameters of the entire model.

[0205] The following evaluation will showcase the experimental results of the ACE framework across multiple medical image analysis tasks, including pre-training performance, transfer learning capabilities, and application effectiveness in classification and segmentation tasks. To validate the effectiveness of ACE, a series of experiments were conducted on different datasets and compared with state-of-the-art methods.

[0206] (1) Properties learned during pre-training

[0207] ACE enhances feature composability: This embodiment investigates the ability of ACE to maintain anatomical composition within its learned embedding space. For example... Figure 3 As shown in -a, patches were randomly extracted from test images in the ChestX-ray14 dataset, and each patch was further decomposed into 2 or 4 non-overlapping sub-patches. Each extracted patch and its sub-patches were resized to a fixed size (i.e., 448×448), and their features were extracted using a pre-trained model from ACE and other benchmark pre-trained models. Subsequently, the cosine similarity between the embedding of each patch and the average embedding of its sub-patches was calculated, and the similarity distribution was visualized using Gaussian kernel density estimation (KDE). The results are as follows. Figure 3 As shown in -b, the distribution of ACE not only exhibits a narrower and taller shape than the baseline model, but the average similarity value between the embeddings of patches and their constituent parts (sub-patches) also shifts towards 1. These observations suggest that ACE effectively integrates compact features while maintaining the compositional integrity of the anatomical structure, echoing its ability to preserve the compositional integrity of the anatomical structure within the learned embedding space.

[0208] ACE enhances the decomposability of features. This embodiment investigates ACE's ability to maintain the decomposability of anatomical structures within its learned embedding space. To this end, the test set of the ChestX-ray14 dataset is first divided into multiple batches, each containing 32 images. Within each batch, a random patch is extracted from each image and labeled as... Figure 4 C in -a j The size is 30%-60% of the original image. Then, the original image and the image with the removed area (marked as X) are compared. j-excised ) and the extracted patch (removing region C) j The pre-trained model of ACE and other benchmark pre-trained models are fed in to extract their features. Finally, the computation is performed. and Cosine similarity between them, to verify Is it close? The results are as follows Figure 4 As shown in -b, ACE significantly outperforms the SSL benchmark model. In particular, ACE achieves a high accuracy of 89.01%, compared to the accuracies of 58.88%, 12.71%, 18.90%, 13.07%, 15.38%, and 3.12% achieved by DINO, PEAC, SelfPatch, DropPos, POPAR, and BYOL, respectively. This significant accuracy difference highlights ACE's superior ability to maintain anatomical decomposition within its learned embedding space.

[0209] ACE provides powerful global image retrieval driven by local features. To explore ACE's ability to capture semantically rich features in its learned embedding space, the test set of the ChestX-ray14 dataset was first divided into multiple batches, each containing 32 images. For each batch, a random image X was selected. j And extract a random patch from it as a query, and mark it as Figure 5 -a contains C. Using ACE's pre-trained model and other benchmark models, query patches are extracted. and features of each complete image in the batch. Then, cosine similarity is calculated between the query patch embedding and the embeddings of all complete images in the batch. The retrieval is considered correct if the highest cosine similarity score corresponds to the query patch and its associated complete image. The results are as follows: Figure 5 As shown in -b, ACE outperforms other SSL benchmark models in retrieval accuracy (94.37%), demonstrating the semantic richness of the representations learned by ACE. This result highlights ACE's potential in clinical practice, enabling accurate identification and retrieval of patients with similar pathological findings based on query patches relevant to specific diseases.

[0210] ACE provides unique anatomical embeddings for different structures. For example... Figure 6 As shown, to investigate the ability of ACE to reflect the locality of anatomical structures in its learned embedding space, a dataset of 1,000 images was compiled from the ChestX-ray14 dataset, with each image annotated by experts with nine different anatomical landmarks. From 1024... 2 448 was extracted from the original image at the specified resolution. 2 Patches of varying sizes are extracted around the location of each marker point, and then latent features for each marker point instance are extracted using a pre-trained model of ACE and other benchmark pre-trained models (untuned). These features are visualized using a t-SNE plot. Figure 6 As shown, the SSL baseline models—DINO, PEAC, POPAR

[36] and DropPos—struggle to generate unique features for different anatomical landmarks, resulting in a blurred embedding space and mixed clusters. However, ACE excels at distinguishing different anatomical landmarks and generating well-separated clusters in the learned embedding space. This highlights ACE's ability to develop rich embedding spaces in which different anatomical structures have unique representations, while the same anatomical structures among patients have very similar embedding representations.

[0211] ACE provides unsupervised cross-patient anatomical mappings. To demonstrate the effectiveness of ACE in capturing diverse anatomical structures, patch-level features were used to query identical anatomical structures across different patients in a zero-shot setting. Specifically, using test images from the ChestX-ray14 dataset, 13 expert-annotated anatomical landmarks were selected, such as... Figure 7 -a is shown. For a given query image, a size of 448 will be extracted at each marker. 2 Patches, these patches come from an initial size of 1024 2 The images. These patches will be fed into ACE's pre-trained backbone network (without fine-tuning) to obtain query features. Then, for other key images, N is extracted using a sliding window. k One patch, window size 448 2 The step size is 8 (boundary patches are zero-padding), and these patches are then input into the ACE backbone network to obtain the feature dictionary of the key image. Finally, regarding E q For each query marker feature in the dataset, we find E k The feature with the closest L2 distance is used to predict the location of the corresponding landmark in the key image. (Result image) Figure 7 As shown, in Figure 7 The -a option plots the predicted and actual marker points, and uses... Figure 7 The box plot in -b analyzes the prediction error for each landmark. The results show that using ACE-encoded features can accurately detect anatomical landmarks, with an average error of 61 pixels (out of 1024) for 13 landmarks. 2 (Image size 1k). The results show that the extracted features reliably represent specific anatomical regions and remain consistent despite significant morphological changes.

[0212] (2) Downstream migration ability of ACE.

[0213] Data efficiency evaluation. This invention analyzes the robustness of ACE representation under limited data conditions. To this end, the pre-trained ACE model is compared with two SSL pre-trained models (POPAR and DINO) by fine-tuning with a small amount of labeled data (2, 5, and 10 samples) on the JSRT-Heart dataset

[38] and with a limited proportion of labeled data (1%, 10%, and 50%) on the SIIM dataset [2]. Figure 8As shown, ACE outperforms the POPAR and DINO models in both heart segmentation and pneumothorax classification tasks with limited data. Particularly in the heart segmentation task, ACE achieved over 91% of its performance on its full training data using only two labeled samples. These results highlight ACE's labeling efficiency, especially in target tasks where labeled data is scarce.

[0214] Fine-tuning evaluation. To investigate the generalization ability of ACE's representation in a fully fine-tuned setting across various downstream tasks, ACE was compared with nine SSL benchmark models with different objectives, covering three classification tasks and four segmentation tasks. Furthermore, performance trained from scratch was included as a lower bound.

[0215] like Figure 9 As shown, the ACE model using the ViT-B backbone network exhibits competitive or superior performance compared to benchmark models (including DINO, SelfPatch, and DropPos) using the same backbone network. Furthermore, ACE using the Swin-B backbone network consistently outperforms BYOL, DINO, and POPAR across all downstream tasks, even though these models also use Swin-B backbone networks. Compared to adaptive methods for medical images, including the visual SSL models POPAR and Adam, and the visual-language SSL methods KAD, ChexZero, and DeViDe, ACE provides best or second-best performance. These results demonstrate the superior transferability and generalization ability of ACE representation across various tasks.

[0216] ACE outperforms various SSL benchmark models with different objectives and backbone networks across multiple classification and segmentation tasks in a fully fine-tuned setting. The best method is highlighted in bold, and the second best method is indicated by underline. Independent two-sample t-tests were performed for each task to compare the best method (bold) with other methods.

[0217] The blue highlighted box indicates results that are not statistically significant at the p=0.05 level.

[0218] (3) Generalization ability of ACE.

[0219] The ACE framework can be seamlessly extended to other imaging modalities. To demonstrate this, ACE was pre-trained on unlabeled fundus images (EyePACS) and fine-tuned on the EyePACS dataset for the classification of diabetic retinopathy. Figure 10 As shown in -a, ACE outperforms the state-of-the-art SSL method DINO, the large-scale pre-trained method LVM-Med, and methods trained from scratch. Furthermore, based on the pre-trained backbone network of ACE, unsupervised anatomical mapping is performed on the FIRE fundus image registration dataset without fine-tuning. Figure 10 As shown in -b, the key points corresponding to the query image can be accurately located in the key image.

[0220] This invention aims to learn visual representations by combining and decomposing anatomical structures in medical images, relying on learning global and local consistency through reliable global representation alignment and correspondence matrix matching. ACE has been rigorously tested through comprehensive experiments on various tasks, demonstrating its outstanding properties and effective transfer capabilities, showing great potential in advancing and interpretable AI applications in medical image analysis.

[0221] The present invention also provides a self-supervised learning system for anatomical consistency of medical images that can be synthesized and decomposed. The self-supervised learning system for anatomical consistency of medical images that can be synthesized and decomposed can be implemented by executing the process steps of the self-supervised learning method for anatomical consistency of medical images that can be synthesized and decomposed. That is, those skilled in the art can understand the self-supervised learning method for anatomical consistency of medical images that can be synthesized and decomposed as a preferred embodiment of the self-supervised learning system for anatomical consistency of medical images that can be synthesized and decomposed.

[0222] Specifically, a self-supervised learning system for synthesizing and decomposing medical images based on anatomical consistency includes:

[0223] Module M1: Generates random image cropping blocks using a grid-based image cropping strategy;

[0224] Module M2: Performs global consistency learning and local consistency learning sequentially based on the image cropping blocks;

[0225] Module M3: Integrates the learning results of global consistency and local consistency, uses the total loss function to capture multi-level information of medical images and optimize model performance.

[0226] The module M1 includes the following sub-modules:

[0227] Module M1.1: Divides the input image into a fixed non-overlapping grid, with each grid measuring m×m;

[0228] Module M1.2: Extracts two randomly cropped views, C1 and C2, from the input image;

[0229] Module M1.3: In the overlapping area of ​​clipping views C1 and C2, further decompose the mesh blocks to obtain sub-blocks of preset size, and complete multi-scale alignment.

[0230] The process of global consistency learning includes:

[0231] Extract consistent semantic features from overlapping regions to capture coarse-grained anatomical structures;

[0232] After being clipped by the grid, the two views C1 and C2 are adjusted to the same size:

[0233]

[0234] Where C is the number of image channels, and H0 and W0 are the height and width of the image, respectively;

[0235] Two random data augmentations, T1 and T2, are applied respectively to generate augmented views:

[0236] x = T1(C1) and x′ = T2(C2);

[0237] Input the augmented view into the student model Teacher Model In the middle, respectively generate embedded representations

[0238] y s ,y t ∈R K×N ;

[0239] Where K is the dimension of the local embedding representation. And (m,m) is the resolution of the sub-block;

[0240] For the overlapping areas of the clipped view, average pooling is used. Generate global embedding representation

[0241]

[0242] Where O1 and O2 are the overlapping regions in C1 and C2, which are normalized to a probability distribution P by the softmax function. s and

[0243]

[0244] Where, τ s It is a temperature parameter used to control the sharpness of the probability distribution;

[0245] Minimize student distribution P s Teacher distribution P t The cross-entropy loss between the two features allows the model to capture globally consistent features. The loss function is as follows:

[0246]

[0247] Where, θ s These are the parameters of the student model, and CE(a,b) = -alogb is the cross-entropy loss.

[0248] The local consistency learning includes local combination and local decomposition to learn fine-grained anatomical features and model anatomical structures.

[0249] The local combination involves combining the embeddings of multiple sub-blocks into a larger overall embedding, encouraging the model to learn the anatomical structure from parts to the whole; for overlapping regions of the cropped image, the embeddings of multiple sub-blocks are consistent with the embedding of their corresponding overall p, and the combined embedding is defined as:

[0250]

[0251] in, It is a feature combiner. This is the student model encoder, where q1, q2, q3, and q4 are sub-blocks. The similarity between matching sub-block embeddings is maximized using the cross-correlation matrix, while the similarity between mismatched sub-block embeddings is minimized. A loss function similar to CLIP is used to guide the model in learning local consistency. When C1 is input into the student model and C2 is input into the teacher model, the resulting embeddings are y... s ,y t ∈R K×N Then y s Input to combiner To combine each 2×2 embedding, the cross-correlation combination matrix is:

[0252]

[0253] Among them, M comp ∈R N×N / 4 T is the transpose of the matrix, (·) represents matrix multiplication, and the sigmoid function is used to restrict the value of each element of the matrix to (0,1); the target matrix T is optimized. comp ∈R N×N / 4 The element value is either 0 or 1, where 1 indicates that the two local embeddings match at their original positions, and 0 indicates that they do not match. The loss function for combined embeddings is calculated by maximizing the similarity of matching embeddings and minimizing the similarity of mismatched embeddings.

[0254]

[0255] The hyperparameter α is used to control the balance between positive and negative samples.

[0256] The local decomposition includes the inverse operation of the local combination process, learning fine-grained anatomical features from the whole to the parts by decomposing a whole image patch into multiple sub-patterns; for a whole image patch p, through a feature decomposer... Decompose it into the embedding of multiple sub-blocks

[0257]

[0258] in, The encoder for the student model;

[0259] Teacher Model Embeddings of corresponding sub-blocks are generated based on the corresponding inputs. To ensure the consistency of the anatomical structure, the similarity between matching sub-block embeddings is maximized by the cross-entropy of the cross-correlation matrix, while the similarity between mismatched embeddings is minimized. The specific loss function is as follows:

[0260]

[0261] Among them, M decomp It is the matching matrix calculated by local decomposition, T decomp α is the target matching matrix, and α is the hyperparameter for balancing positive and negative samples.

[0262] The total loss function integrates the global consistency loss L global The local consistency loss is defined as follows:

[0263]

[0264] Among them, L comp For the combination loss, L decomp To decompose the loss, λ1, λ2, and λ3 are weighting parameters used to balance the different loss terms.

[0265] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, making the system and its various devices, modules, and units of this invention function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered as a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0266] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A self-supervised learning method for anatomical consistency in medical image synthesis and decomposition, characterized in that, include: Step S1: Generate random image cropping blocks using a grid-based image cropping strategy; Step S2: Perform global consistency learning and local consistency learning sequentially based on the image cropping blocks; Step S3: Integrate the learning results of global consistency and local consistency, use the total loss function to capture multi-level information of medical images and optimize model performance; The local consistency learning includes local combination and local decomposition to learn fine-grained anatomical features and model anatomical structures. The local combination involves combining the embeddings of multiple sub-blocks into a larger overall embedding, encouraging the model to learn the anatomical structure from parts to the whole; for overlapping regions of the cropped image, the embeddings of multiple sub-blocks are consistent with the embedding of their corresponding overall p, and the combined embedding is defined as: in, It is a feature combiner. This is the student model encoder, where q1, q2, q3, and q4 are sub-blocks. The similarity between matching sub-block embeddings is maximized using the cross-correlation matrix, while the similarity between mismatched sub-block embeddings is minimized. A loss function similar to CLIP is used to guide the model in learning local consistency. When C1 is input into the student model and C2 is input into the teacher model, the resulting embeddings are y... s ,y t ∈R K×N Then y s Input to combiner To combine each 2×2 embedding, the cross-correlation combination matrix is: Among them, M comp ∈R N×N / 4 T is the transpose of the matrix, (·) represents matrix multiplication, and the sigmoid function is used to restrict the value of each element of the matrix to (0,1); the target matrix T is optimized. comp ∈R N×N / 4 The element value is either 0 or 1, where 1 indicates that the two local embeddings match at their original positions, and 0 indicates that they do not match. The loss function for combined embeddings is calculated by maximizing the similarity of matching embeddings and minimizing the similarity of mismatched embeddings. Among them, the hyperparameter α is used to control the balance between positive and negative samples; The local decomposition includes the inverse operation of the local combination process, learning fine-grained anatomical features from the whole to the parts by decomposing a whole image patch into multiple sub-patterns; for a whole image patch p, through a feature decomposer... Decompose it into the embedding of multiple sub-blocks in, The encoder for the student model; Teacher Model Embeddings of corresponding sub-blocks are generated based on the corresponding inputs. To ensure the consistency of the anatomical structure, the similarity between matching sub-block embeddings is maximized by the cross-entropy of the cross-correlation matrix, while the similarity between mismatched embeddings is minimized. The specific loss function is as follows: Among them, M decomp It is the matching matrix calculated by local decomposition, T decomp α is the target matching matrix, and α is the hyperparameter that balances positive and negative samples; The total loss function integrates the global consistency loss L global The local consistency loss is defined as follows: Among them, L comp For the combination loss, L decomp To decompose the loss, λ1, λ2, and λ3 are weighting parameters used to balance the different loss terms.

2. The self-supervised learning method for anatomical consistency of medical images that can be synthesized and decomposed according to claim 1, characterized in that, Step S1 includes the following sub-steps: Step S1.1: Divide the input image into a fixed non-overlapping grid, with each grid having a size of m×m; Step S1.2: Extract two randomly cropped views C1 and C2 from the input image; Step S1.3: In the overlapping area of ​​cropped views C1 and C2, further decompose the grid blocks to obtain sub-blocks of preset size, and complete multi-scale alignment.

3. The self-supervised learning method for anatomical consistency of medical images that can be synthesized and decomposed according to claim 2, characterized in that, The process of global consistency learning includes: Extract consistent semantic features from overlapping regions to capture coarse-grained anatomical structures; After being clipped by the grid, the two views C1 and C2 are adjusted to the same size: Where C is the number of image channels, and H0 and W0 are the height and width of the image, respectively; Two random data augmentations, T1 and T2, are applied respectively to generate augmented views: x = T1(C1) and x ′ =T2(C2); Input the augmented view into the student model Teacher Model In the middle, respectively generate embedded representations and s ,and t ∈R K×N ; Where K is the dimension of the local embedding representation. And (m,m) is the resolution of the sub-block; For the overlapping areas of the clipped view, average pooling is used. Generate global embedding representation Where O1 and O2 are the overlapping regions in C1 and C2, which are normalized to a probability distribution P by the softmax function. s and Where, τ s It is a temperature parameter used to control the sharpness of the probability distribution; Minimize student distribution P s Teacher distribution P t The cross-entropy loss between the two features allows the model to capture globally consistent features. The loss function is as follows: Where, θ s These are the parameters of the student model, and CE(a,b) = -alogb is the cross-entropy loss.

4. A self-supervised learning system for synthesizing and decomposing medical images based on anatomical consistency, characterized in that, include: Module M1: Generates random image cropping blocks using a grid-based image cropping strategy; Module M2: Performs global consistency learning and local consistency learning sequentially based on the image cropping blocks; Module M3: Integrates the learning results of global consistency and local consistency, uses the total loss function to capture multi-level information of medical images and optimize model performance; The local consistency learning includes local combination and local decomposition to learn fine-grained anatomical features and model anatomical structures. The local combination involves combining the embeddings of multiple sub-blocks into a larger overall embedding, encouraging the model to learn the anatomical structure from parts to the whole; for overlapping regions of the cropped image, the embeddings of multiple sub-blocks are consistent with the embedding of their corresponding overall p, and the combined embedding is defined as: in, It is a feature combiner. This is the student model encoder, where q1, q2, q3, and q4 are sub-blocks. The similarity between matching sub-block embeddings is maximized using the cross-correlation matrix, while the similarity between mismatched sub-block embeddings is minimized. A loss function similar to CLIP is used to guide the model in learning local consistency. When C1 is input into the student model and C2 is input into the teacher model, the resulting embeddings are y... s ,y t ∈R K×N Then y s Input to combiner To combine each 2×2 embedding, the cross-correlation combination matrix is: Among them, M comp ∈R N×N / 4 T is the transpose of the matrix, (·) represents matrix multiplication, and the sigmoid function is used to restrict the value of each element of the matrix to (0,1); the target matrix T is optimized. comp ∈R N×N / 4 The element value is either 0 or 1, where 1 indicates that the two local embeddings match at their original positions, and 0 indicates that they do not match. The loss function for combined embeddings is calculated by maximizing the similarity of matching embeddings and minimizing the similarity of mismatched embeddings. Among them, the hyperparameter α is used to control the balance between positive and negative samples; The local decomposition includes the inverse operation of the local combination process, learning fine-grained anatomical features from the whole to the parts by decomposing a whole image patch into multiple sub-patterns; for a whole image patch p, through a feature decomposer... Decompose it into the embedding of multiple sub-blocks in, The encoder for the student model; Teacher Model Embeddings of corresponding sub-blocks are generated based on the corresponding inputs. To ensure the consistency of the anatomical structure, the similarity between matching sub-block embeddings is maximized by the cross-entropy of the cross-correlation matrix, while the similarity between mismatched embeddings is minimized. The specific loss function is as follows: Among them, M decomp It is the matching matrix calculated by local decomposition, T decomp α is the target matching matrix, and α is the hyperparameter that balances positive and negative samples; The total loss function integrates the global consistency loss L global The local consistency loss is defined as follows: Among them, L comp For the combination loss, L decomp To decompose the loss, λ1, λ2, and λ3 are weighting parameters used to balance the different loss terms.

5. The self-supervised learning system for anatomical consistency of medical images that can be synthesized and decomposed according to claim 4, characterized in that, The module M1 Includes the following sub-modules: Module M1.1: Divides the input image into a fixed non-overlapping grid, with each grid measuring m×m; Module M1.2: Extracts two randomly cropped views, C1 and C2, from the input image; Module M1.3: In the overlapping area of ​​clipping views C1 and C2, further decompose the mesh blocks to obtain sub-blocks of preset size, and complete multi-scale alignment.

6. The medical image synthesis and decomposability anatomical consistency self-supervised learning system according to claim 5, characterized in that, The process of global consistency learning includes: Extract consistent semantic features from overlapping regions to capture coarse-grained anatomical structures; After being clipped by the grid, the two views C1 and C2 are adjusted to the same size: Where C is the number of image channels, and H0 and W0 are the height and width of the image, respectively; Two random data augmentations, T1 and T2, are applied respectively to generate augmented views: x = T1(C1) and x ′ =T2(C2); Input the augmented view into the student model f θs And teacher model f θt In the middle, respectively generate embedded representations and s ,and t ∈R K×N ; Where K is the dimension of the local embedding representation. And (m,m) is the resolution of the sub-block; For the overlapping areas of the clipped view, average pooling is used. Generate global embedding representation Where O1 and O2 are the overlapping regions in C1 and C2, which are normalized to a probability distribution P by the softmax function. s and Where, τ s It is a temperature parameter used to control the sharpness of the probability distribution; Minimize student distribution P s Teacher distribution P t The cross-entropy loss between the two features allows the model to capture globally consistent features. The loss function is as follows: Where, θ s These are the parameters of the student model, and CE(a,b) = -alogb is the cross-entropy loss.

Images