Medical image style transfer method, system, device, medium and program product

Abstract

The application provides a medical image style transfer method, system, device, medium and program product, and relates to the fields of medical image processing and computer vision. The application comprises: performing first weighted processing on a source medical image and guide information by using an image language model to obtain source medical image features and guide information features, fusing the source medical image features and the guide information features to obtain target condition features; performing preprocessing on the source medical image to obtain enhanced image features; fusing the target condition features and the enhanced image features to obtain fusion features; and performing second weighted processing on the fusion features to obtain a target style medical image. The application has the advantages of high flexibility, high style fidelity, good anatomical structure retention and superior performance.

Description

Technical Field

[0001] This application relates to the fields of medical image processing and computer vision, and in particular to methods, systems, devices, media, and program products for medical image style transfer. Background Technology

[0002] In clinical practice and multicenter studies, medical images are typically acquired by scanners from different manufacturers (such as Siemens, Philips, GE, etc.) under different acquisition protocols. Due to hardware differences and variations in physical acquisition parameters (repetition time (TR), echo time (TE), reversal time (TI), flip angle (FA), etc.), images often exhibit significant style inconsistencies (such as differences in contrast, brightness, and signal-to-noise ratio). This "domain shift" severely hinders the universality of subsequent clinical diagnostic and automated analysis algorithms. Existing medical image style transfer methods mainly suffer from the following limitations:

[0003] 1) Lack of parameter guidance: Most methods (such as histogram matching or traditional GAN ​​methods) fail to explicitly utilize physical acquisition parameters (TR, TE, etc.) to guide the image style transfer process, making it difficult for the model to adapt to diverse acquisition settings;

[0004] 2) Single and fixed target domain: Existing models can typically only map images to a predefined, fixed target domain, failing to flexibly support arbitrarily specified target styles. Retraining the model for different targets usually requires retraining, which is inefficient.

[0005] 3) Lack of global style constraints: Existing methods mainly rely on pixel-level supervision (such as voxel-level loss or adversarial loss), lacking robust global constraints to ensure that the generated images strictly conform to the target style, resulting in insufficient fidelity of anatomical structures or style consistency.

[0006] Therefore, there is an urgent need for a method that can uniformly process multiple source domains and flexibly generate high-quality images based on specific acquisition parameters and reference images. Summary of the Invention

[0007] In view of the shortcomings of the prior art described above, the purpose of this application is to provide a medical image style transfer method, system, device, medium and program product to solve the technical problems of the prior art in medical image style transfer, such as lack of explicit guidance on physical acquisition parameters, single and fixed target style domain that is difficult to flexibly adapt to arbitrary targets, and lack of global style constraints that leads to insufficient fidelity of anatomical structure and style consistency.

[0008] To achieve the above and other related objectives, a first aspect of this application provides a medical image style transfer method, comprising: performing a first weighted processing on a source medical image and guidance information using an image language model to obtain source medical image features and guidance information features; fusing the source medical image features and guidance information features to obtain target conditional features; preprocessing the source medical image to obtain enhanced image features; fusing the target conditional features and enhanced image features to obtain fused features; and performing a second weighted processing on the fused features to obtain a target style medical image.

[0009] In some embodiments of the first aspect of this application, the first weighted processing of the source medical image and guidance information to obtain target conditional features includes: fusing the guidance information with learnable text information to obtain parametric conditional features; extracting features from the source medical image to obtain reference image style features; and fusing the parametric conditional features with the reference image style features to obtain target conditional features.

[0010] In some embodiments of the first aspect of this application, the target condition features and enhanced image features are fused to obtain fused features, wherein the fusion employs a cross-attention mechanism; wherein the key matrix and value matrix are derived from the enhanced image features, and the query matrix is ​​derived from the target condition features.

[0011] In some embodiments of the first aspect of this application, in the step of fusing the target condition features and the enhanced image features to obtain fused features, the enhanced image features are first divided, and then fused with the target condition features.

[0012] In some embodiments of the first aspect of this application, the preprocessing includes any of the following: applying a random bias field to simulate magnetic field inhomogeneity, performing random downsampling to simulate different resolutions, applying gamma correction and adjusting contrast and brightness.

[0013] To achieve the above and other related objectives, a second aspect of this application provides a model training method for training an image language model as described in the first aspect of this application, comprising: extracting features from a source medical image, guidance information, and a target style medical image; calculating the similarity between the source medical image and the target style medical image, and the similarity between the guidance information and the target style medical image, to obtain an image-image similarity matrix and an image-text similarity matrix; and training the feature extraction processes of the source medical image, guidance information, and target style medical image based on the loss between the image-image similarity matrix, the image-text similarity matrix, and the standard diagonal matrix.

[0014] To achieve the above and other related objectives, a third aspect of this application provides a medical image style transfer system, comprising: a first weighting module, configured to perform a first weighting process on a source medical image and guidance information using an image language model to obtain source medical image features and guidance information features, and to fuse the source medical image features and guidance information features to obtain target conditional features; a preprocessing module, configured to preprocess the source medical image to obtain enhanced image features; a second weighting module, configured to fuse the target conditional features and enhanced image features to obtain fused features; and a style transfer module, configured to perform a second weighting process on the fused features to obtain a target style medical image.

[0015] To achieve the above and other related objectives, a fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the medical image style transfer method.

[0016] To achieve the above and other related objectives, a fifth aspect of this application provides a computer program product comprising computer program code that, when executed on a computer, causes the computer to implement the medical image style transfer method.

[0017] To achieve the above and other related objectives, a sixth aspect of this application provides a computer device / equipment / system / electronic terminal, including a memory, a processor, and a computer program stored in the memory; the processor executes the computer program to implement the medical image style transfer method.

[0018] As described above, the medical image style transfer method, system, apparatus, medium, and program product of this application have the following beneficial effects:

[0019] 1) High flexibility: A single model can support conversion to any specified acquisition parameters or reference style without the need for retraining for a new target;

[0020] 2) High style fidelity: By introducing multimodal contrast loss, the generated image is forced to conform to the target parameter setting from the global feature level, which is more effective than simple pixel-level loss;

[0021] 3) Good preservation of anatomical structure: Combining voxel-level loss and structural similarity loss, the anatomical details of medical images are preserved to the greatest extent while adjusting the style;

[0022] 4) Superior performance: Validated on multi-scanner, multi-site travel subject datasets, the metrics of this method are superior to those of existing mainstream methods. Attached Figure Description

[0023] Figure 1 A schematic diagram of an implementation environment provided for an embodiment of this application.

[0024] Figure 2 This is an architecture diagram of the medical image style transfer method provided in the embodiments of this application.

[0025] Figure 3 This is a flowchart illustrating the medical image style transfer method provided in an embodiment of this application.

[0026] Figure 4 This is a schematic diagram of the sub-process of step 1) in the medical image style transfer method provided in the embodiments of this application.

[0027] Figure 5 This is a schematic diagram illustrating the training process of the image language model provided in an embodiment of this application.

[0028] Figure 6 This is a statistical distribution diagram of the multicenter MRI dataset in this embodiment of the application.

[0029] Figure 7 This is a qualitative comparison diagram of cross-scanner style transfer results in the embodiments of this application.

[0030] Figure 8 A framework diagram of a medical image style transfer system provided in an embodiment of this application.

[0031] Figure 9 A frame diagram of a computer device provided for an embodiment of this application. Detailed Implementation

[0032] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0033] Before providing a further detailed description of this application, the nouns and terms used in the embodiments of this application are explained, and the nouns and terms used in the embodiments of this application shall be interpreted as follows:

[0034] To address the technical problems described above, this application introduces a novel method that uses deep learning to convert source medical images into specified target medical images, particularly magnetic resonance imaging (MRI) images. However, this application is not limited to this and can also be applied to other medical images, such as CT images, PET images, ultrasound images, etc. It should be understood that the method provided in this application, in addition to its application in the medical field, can also be used in industrial inspection, video surveillance, military defense, aerospace, autonomous driving, and other fields, and this application does not limit its application in these areas.

[0035] The implementation environment of the embodiments of this application is described below. Figure 1 This is a schematic diagram of an implementation environment provided for an embodiment of this application, such as... Figure 1 As shown, this implementation environment includes a terminal device 101 and a server 102. The terminal device 101 and the server 102 are connected via wired or wireless means.

[0036] Specifically, the terminal device 101 in this embodiment of the application is equipped with a client. The user can input source medical images and guidance information, such as magnetic resonance images and guidance information for the desired target magnetic resonance image, on the client. The server 102 acts as a server, performing style transfer on the source medical images and guidance information input by the user on the client, and sending the style-transferred magnetic resonance images to the client for display.

[0037] In some embodiments, such as Figure 1 As shown, the server 102 in this embodiment includes a medical image style transfer model, which is used to perform style transfer on the source medical image as input and output the style-transferred medical image. It should be noted that in this embodiment, before using the medical image style transfer model to perform style transfer on the source medical image as input, the medical image style transfer model is first trained.

[0038] In some embodiments, the medical image style transfer model described above is trained by server 102. For example, an object sends a source medical image, guidance information, and a style-transferred medical image corresponding to the source medical image to server 102 via terminal device 101. Server 102 stores the medical image style transfer model to be trained. Thus, server 102 can process the source medical image, guidance information, and style-transferred medical image sent by terminal device 101 using the medical image style transfer model, determine the loss of the medical image style transfer model, and train the medical image style transfer model based on the loss. Then, server 102 saves the trained medical image style transfer model or saves it on other storage devices.

[0039] In some embodiments, the medical image style transfer model described above may also be trained on other computing devices, and the trained medical image style transfer model may be sent to server 102 for storage or stored on other storage devices.

[0040] In the actual diagnostic process of the medical image style transfer model, the patient inputs the source medical image and guidance information on the client side of terminal device 101. Terminal device 101 then forwards the source medical image and guidance information to server 102. Server 102 inputs the source medical image and guidance information into the medical image style transfer model, enabling the model to perform style transfer based on the source medical image and guidance information, resulting in a style-transferred medical image. Finally, server 102 sends the style-transferred medical image result generated by the medical image style transfer model to terminal device 101 for display, allowing the patient to obtain medical images with different styles.

[0041] In some embodiments, the terminal device 101 includes, but is not limited to, desktop computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices may include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle systems. Portable wearable devices may include smartwatches, smart bracelets, and head-mounted devices. Terminal devices are often equipped with a display device, which may be a monitor, display screen, touchscreen, etc., and the touchscreen may be a touchscreen, touch panel, etc.

[0042] In some embodiments, the server 102 described above can be one or more servers. When there are multiple servers, at least two servers are used to provide different services, and / or at least two servers are used to provide the same service, such as providing the same service in a load-balanced manner. This application embodiment does not limit this. The server described above can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The server can also become a node in a blockchain.

[0043] It should be noted that the implementation environment of this application embodiment includes, but is not limited to, Figure 1 As shown.

[0044] The technical solutions of the embodiments of this application will be described in detail below through examples. First, the inference process of the medical image style transfer model will be introduced. Figure 2This is a diagram illustrating the overall architecture of a medical image style transfer model provided in an embodiment of this application. It shows the complete structure of the model during the training phase; during the inference phase, [the model will...]. Figure 2 (A) Simply replace the Dataaugmentation module with a real source medical image as input. Figure 2 (B) Prompt and image-style-conditioned harmonization (conditional generation module) reveals the process of model inference. Figure 2 (C) The Multi-modal contrastive loss module reveals the multi-modal contrastive loss applied to the Text encoder and Image encoder, with the weights of the Text encoder and Image encoder locked throughout the process. Based on this, the specific process of the inference stage is described below with reference to the architecture shown in Figure 2. The execution entity of this application embodiment can be a device with model inference capabilities, such as a model inference device. This model inference device can be the one described above. Figure 1 The server 102 or terminal device 101 shown can also be Figure 1 The system consists of servers 102 or 101 as shown. For ease of description, the following embodiments use electronic devices as examples to illustrate the methods of this application. Figure 2 As shown, the model reasoning method in this application embodiment includes:

[0045] Step 1) Perform a first weighted processing on the source medical image and guidance information to obtain reference image style features and parameter condition features, and fuse the reference image style features and parameter condition features to obtain target condition features.

[0046] Specifically, the first weighted processing of the source medical image can employ a source medical image encoder, and the first weighted processing of the guidance information can employ a guidance information encoder. Both the source medical image encoder and the guidance information encoder can utilize a pre-trained Transformer architecture, with parameters frozen during the training of the medical image style transfer model.

[0047] The reference image style features and parameter condition features are typically represented by feature vectors or feature matrices.

[0048] Both the source medical image encoder and the guidance information encoder are pre-trained. To better understand the roles of the source medical image encoder and the guidance information encoder, the training methods for these two encoders will be explained in detail first, such as... Figure 2 and Figure 5 As shown, the training method includes:

[0049] Training step 1) Extract features from the source medical image, guidance information and target style medical image respectively.

[0050] Specifically, a source medical image encoder is used to extract features from the source medical image, a guidance information encoder is used to extract features from the guidance information, and a target style medical image encoder is used to extract features from the target style medical image. All three encoders can use the architecture described above, and the initial parameters (such as the value of the convolutional kernel) can be set randomly.

[0051] Training step 2) Calculate the similarity between the source medical image and the target style medical image, and the similarity between the guidance information and the target style medical image, to obtain the image-image similarity matrix and the image-text similarity matrix.

[0052] Taking the CLIP architecture as an example: In the image-image similarity matrix, rows represent source medical image features, and columns represent target style medical image features, with the number of rows and columns being equal. Therefore, in the image-image similarity matrix, each point represents the similarity between the I-th source medical image feature and the I'-th target style medical image feature, and the value can be calculated using cosine similarity.

[0053] Similarly, in the image-text similarity matrix, rows represent guiding information features, and columns represent target style medical image features, with the number of rows and columns being relatively equal. Therefore, in the image-text similarity matrix, each point represents the similarity between the T-th guiding information feature and the I'-th target style medical image feature, and the value can be calculated using cosine similarity.

[0054] Training step 3) Based on the loss between the image-image similarity matrix, the image-text similarity matrix and the standard diagonal matrix, the feature extraction process of the source medical image, the guidance information and the target style medical image is trained respectively.

[0055] The standard diagonal matrix has the same number of rows and columns as the image-to-image similarity matrix and the image-to-text similarity matrix. Except for the values ​​on the diagonal, all other values ​​in the standard diagonal matrix are 0. That is, the standard diagonal matrix is ​​used to shorten the distance between paired sample pairs and widen the distance between unpaired sample pairs, constraining and aligning the source medical image and guiding information to the target style medical image. In the image-to-image similarity matrix, the standard diagonal matrix is ​​used to improve the consistency between the generated image and the target reference image in the feature space; in the image-to-text similarity matrix, the standard diagonal matrix is ​​used to improve the consistency between the target parameter text and the target reference image in the feature space.

[0056] The loss generated by the image-image similarity matrix and the standard diagonal matrix is ​​used to optimize the parameters of the source medical image encoder and the target medical image encoder, and the loss generated by the image-text similarity matrix and the standard diagonal matrix is ​​used to optimize the parameters of the guidance information encoder and the target medical image encoder.

[0057] It should be emphasized that the target style medical image encoder is discarded after training is completed, while the source medical image encoder and the guidance information encoder have their parameters locked after training is completed, and are used to perform the first weighted processing on the source medical image and guidance information, respectively.

[0058] like Figure 4 As shown, step 1) also includes the following steps:

[0059] Step 1-1) Fuse the guidance information features with the learnable text information to obtain parameter condition features.

[0060] The guidance information features are obtained through a guidance information encoder, which is the same one used during training, typically a Transformer architecture. The learnable text information is represented as a set of fixed-length trainable context embedding vectors, whose parameters are randomly initialized and continuously optimized during training to enhance the model's ability to express and control the collected parameter conditions. When fusing the guidance information features and learnable text information, they can first be concatenated along the sequence dimension to obtain the enhanced prompt sequence. For example, the guidance information features and learnable text information can be expressed as feature matrices with consistent length and width, and then concatenated along the sequence dimension. Subsequently, the concatenated feature matrix can be input into a linear adapter, which typically includes a dimension-reducing linear layer, an activation layer, and a dimension-increasing linear layer. After entering the linear adapter, the concatenated feature matrix undergoes dimensionality reduction, followed by activation using a function such as ReLU, and then is restored to the input dimension. The final output is the parameter-conditional feature.

[0061] Steps 1-2) Extract features from the source medical image to obtain reference image style features.

[0062] The reference image style features are obtained through a source medical image encoder, which is the same source medical image encoder used during training as described above, and is usually a Transformer architecture.

[0063] Steps 1-3) fuse the parameter condition features with the reference image style features to obtain the target condition features.

[0064] The fusion of the parameter condition features and the reference image style features can be achieved by stitching them together and then performing linear projection. That is, the parameter condition features and the reference image style features are stitched together in the feature dimension, and then projected through a learnable linear layer to obtain the target condition features. The dimension of the target condition features should be the same as the dimension of the reference image style features and the parameter condition features.

[0065] Therefore, step 1) realizes the construction and representation of target conditions: that is, for each target style conversion, the target acquisition parameters are organized into parameter text according to the preset template, and a reference image from the target acquisition settings is selected; the "reference image style features" are generated by the source medical image encoder, the "parameter condition features" are generated by the guide information encoder combined with the adapter, and then fused to form the "target condition features", which serve as the unified control signal for the subsequent generation process.

[0066] Step 2) Preprocess the source medical image to obtain an enhanced image.

[0067] The preprocessing is intended to enhance the style of the input source medical image.

[0068] The preprocessing may include the following operations: applying a random bias field to simulate magnetic field inhomogeneity, performing random downsampling to simulate different resolutions, applying gamma correction and adjusting contrast and brightness, etc.

[0069] Step 3) Fuse the target condition features and the enhanced image features to obtain fused features.

[0070] The fusion of the target conditional features and the enhanced image features can employ a cross-attention mechanism. The core advantage of the cross-attention mechanism lies in its ability to allow the model to establish dynamic and selective information associations between different sequences or modalities.

[0071] To address the issue of excessive memory consumption when performing attention calculations on every pixel of the enhanced image features, the enhanced image features are first divided into blocks, and then fused with the target conditional features in blocks. Block division not only solves the memory consumption problem but also implicitly preserves the local structure. Pixels within each block are fused into an initial representation after projection, preserving local correlations.

[0072] In one example embodiment, cross-attention is used to reconstruct the target conditional features and enhanced image features. The target conditional features are used as the query, and the enhanced image features are used as the key and value. Attention weights are calculated and weighted. Preferably, the enhanced image features can first be segmented into small blocks and mapped to latent feature sequences. Then, cross-attention is used to generate and reconstruct the target conditional features, enabling dynamic and selective information associations to be established between different sequences or different modalities.

[0073] Therefore, step 3) achieves biconditional guided style transfer generation: the source medical image is input into the conditional generator (using a cross-attention mechanism). During multi-level feature updates, the conditional generator receives control from the "target conditional features" via the cross-attention mechanism, gradually transforming the output image towards the target style while suppressing unnecessary anatomical changes using structure-fidelity-related constraints.

[0074] Step 4) Perform a second weighting process on the fused features to obtain the target style medical image.

[0075] The second weighting process includes: applying a linear voxel regression layer to each attention-weighted feature block. This linear voxel regression layer is essentially a fully connected layer (linear layer) without an activation function; its output is the predicted value of all voxels within the block, and pixel value prediction is performed block by block. Subsequently, the feature block sequence is rearranged and restored to the image space grid through a de-blocking operation to obtain the target style medical image. The parameters of the linear voxel regression layer are learnable weights. This design is highly suitable for high-resolution, 3D structured data like medical images, utilizing the global modeling capabilities of attention while maintaining efficient reconstruction of local details.

[0076] For ease of understanding, combined with Figure 1 The flowchart of the Traditional Chinese Medicine image style transfer method fully illustrates the training method of the medical image style transfer model (referred to as such in this embodiment) used in the embodiments of this application:

[0077] Training Step 1) Input the source medical image and guidance information into the medical image style transfer model, and use the reference target style medical image as the training target;

[0078] Training step 2) Extract features from the guidance information to obtain guidance information features; then use a linear adapter to fuse the guidance information features with learnable text information to obtain parameter condition features; wherein, the linear adapter uses learnable weights;

[0079] Training step 3) Extract features from the source medical image to obtain reference image style features; fuse the parameter condition features with the reference image style features to obtain target condition features;

[0080] Training step 4) Preprocess the source medical image to obtain enhanced image features; use a cross-attention mechanism to fuse the target conditional features and enhanced image features to obtain fused features; wherein, the cross-attention mechanism uses learnable weights;

[0081] Training step 5) Perform a second weighting process on the fused features to obtain the target style medical image; optimize the learnable parameters in the linear adapter and cross-attention mechanism by referencing the loss between the target style medical images.

[0082] Therefore, this application does not rely on strictly paired data during the training phase. Instead, it applies style enhancement to the source image to simulate domain shift and learns to recover the target style under given target conditions. Furthermore, based on the constraints of structure fidelity loss (i.e., the loss during medical image style transfer training, including voxel-level loss and structural similarity loss), this application introduces multimodal contrast constraints (image-text alignment loss and image-reference image alignment loss) in the CLIP feature space. These constraints enhance the consistency between the generated image and the target parameter text prompt, as well as the style consistency with the target reference image, thereby improving the consistency between the generated image and the target acquisition style at the global feature level.

[0083] Example

[0084] Inputs: 1) Source MRI image, which is the raw brain MRI image to be processed, such as a T1-weighted image; 2) Target acquisition settings, which is a description of the target scan parameters that the user expects to convert to, including the scanner manufacturer, specific model, magnetic field strength, and physical acquisition parameters such as TR, TE, TI, and FA; 3) Target reference image, which is an MRI image acquired under the above target acquisition settings and from different subjects, used as a visual reference for the target style.

[0085] Step 1: Data Preprocessing and Style Enhancement (Training Phase). To address the issue of unpaired multi-center data, style enhancement is first performed on the input source images during the training phase. Specifically, style enhancement operators simulate imaging differences from different scanners. These operations include: applying a random bias field to simulate magnetic field inhomogeneity, performing random downsampling to simulate different resolutions, and applying gamma correction and adjusting contrast and brightness. The enhanced image is then used as input to the generator, and the network is trained to restore and transform it to the target style.

[0086] Step 2: Constructing the Joint Target Conditional Vector. A pre-trained MRI-CLIP model is used to extract target conditional features to guide the generation process. First, the target acquisition settings information is organized into parameter text prompts according to a preset template and input into the MRI-CLIP text encoder to obtain parameter text features. To improve the expressive power and control accuracy of the parameter conditions, the parameter text features are fused with a learnable context vector and further refined through a linear adapter layer to obtain parameter condition features. Second, the target reference image is input into the MRI-CLIP image encoder to obtain reference image style features, which are used to characterize the global appearance attributes of the target domain. Finally, the parameter condition features and reference image style features are fused and obtained through linear projection to obtain the joint target conditional vector.

[0087] Step 3: Generation and Reconstruction Based on Cross-Attention. The image from Step 1 is segmented into small blocks and mapped to a sequence of latent features. Style information is injected into the content features using a cross-attention mechanism: the joint target conditional vector obtained in Step 2 is used as the query, and the latent image features are used as keys and values. Attention weights are calculated and weighted. Finally, a linear voxel regression layer is applied to each feature block to predict pixel values, and then the image is restored to the image space through a de-segmentation operation.

[0088] Step 4: Loss Optimization. To ensure the output image conforms to the target parameters and style, the structural fidelity loss and multimodal contrast loss are jointly optimized. The structural fidelity loss operates in the pixel space and includes pixel-level absolute error loss between the generated and source images, as well as structural similarity loss. This constrains the generated image to maintain anatomical consistency with the source image at the low-level feature level, suppressing unnecessary structural deformation. The multimodal contrast loss operates in the MRI-CLIP feature space and includes two components: First, the image-cue contrast loss aligns the generated image with its corresponding target acquisition parameter text cues in the MRI-CLIP embedding space. In each training batch, the generated image and its corresponding parameter cues constitute positive sample pairs, while the remaining cue embeddings within the batch are used as negative samples, calculated using contrastive cross-entropy loss. Second, the image-reference image contrast loss aligns the generated image with reference images from different subjects under the target acquisition settings in the MRI-CLIP embedding space. This is also calculated using contrastive cross-entropy loss, with the remaining target setting reference image embeddings within the batch used as negative samples. Both approaches anchor the target style semantically: parameter text prompts provide quantifiable constraints on acquisition parameters, while reference images provide visual appearance constraints. These two constraints reinforce the consistency between the generated image and the target acquisition style from different perspectives, compensating for the insufficient global style constraints when relying solely on pixel-level losses. The overall training loss is a weighted sum of the above losses, where the weight coefficients balance the contributions of anatomical fidelity and target style consistency.

[0089] Output: Style-transferred MRI image: Outputs a high-quality MRI image that preserves the original subject's anatomy while retaining the imaging style of the specified target scanner (such as specific contrast distribution and texture features). This image eliminates non-biological variations caused by device differences and can be directly used for downstream automated analysis or clinical diagnosis.

[0090] Figure 2 The diagram shown is an architecture diagram of the medical image style transfer model provided in this implementation. Figure 2 (A) represents the data augmentation module, which illustrates the application of style enhancements such as bias field, downsampling, and gamma correction to the source image to simulate differences in multi-center acquisition. Figure 2 (B) represents the conditional generation module, which illustrates the extraction of parameter text features and reference image style features through the CLIP model, and the use of adapter and cross-attention mechanisms to guide the generator to complete the target style transfer. Figure 2 (C) represents the multimodal contrast constraint module, which illustrates the calculation of the "generated image - parameter text" alignment loss and the "generated image - reference image" alignment loss in the CLIP feature space to enhance the consistency of the target style.

[0091] like Figure 6 The figure shown is a statistical distribution chart of a multicenter MRI dataset. Figure 6 The leftmost pie chart represents the proportion of data source sites. Figure 6 The pie chart in the center represents the gender distribution of the participants. Figure 6 The pie chart on the far right shows the age distribution of the subjects; the Sankey diagram on the right shows the distribution of different scanner manufacturers (Siemens, UIH, GE, Philips) and their specific models in the dataset.

[0092] Figure 7 This is a multi-model qualitative comparison chart showing style transfer results across scanners. The example demonstrates the conversion effect from Siemens acquisition conditions to GE acquisition conditions; each row corresponds to axial, sagittal, and coronal slices, respectively. Figure 7 The comparison images show the source image (Source), the target reference image (Target), the output (Ours) of the method described in this application, and the generation results of comparison methods such as HM, SSIMH, CycleGAN, ResViT, and DeepHarmony. Color-coded images represent the differences between the outputs of each method and the target style. The results show that the output (Ours) of this application has the smallest difference from the target reference image (Target), indicating the best performance.

[0093] The foregoing has shown and described the basic principles and main features of the present invention. The above embodiments do not limit the present invention in any way, and all technical solutions obtained by equivalent substitution or equivalent transformation fall within the protection scope of the present invention.

[0094] It should also be noted that in the embodiments of this application, the terms "first" and "second" are used to distinguish identical or similar items with essentially the same function and effect.

[0095] It should be noted that, in the embodiments of this application, the words "exemplary" or "for example" indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0096] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0097] Figure 8 This is a schematic block diagram of the medical image style transfer system provided in the embodiments of this application. Figure 8 As shown, the system includes: a first weighting module 51, used to perform a first weighting process on the source medical image and guidance information using an image language model to obtain source medical image features and guidance information features, and to fuse the source medical image features and guidance information features to obtain a first fused feature; a preprocessing module 52, used to preprocess the source medical image to obtain enhanced image features; a second weighting module 53, used to fuse the target condition features and enhanced image features to obtain fused features; and a style transfer module 54, used to perform a second weighting process on the fused features to obtain a target style medical image.

[0098] It should be understood that the specific processes by which each module performs the corresponding steps described above have been detailed in the above method embodiments, and will not be repeated here for the sake of brevity. It should also be understood that the module division in the embodiments of this application is illustrative and merely a logical functional division; other division methods may exist in actual implementation. Furthermore, the functional modules in the various embodiments of this application can be integrated into a single processor, exist as separate physical entities, or have two or more modules integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0099] Figure 9 This is a schematic block diagram of a computer device provided in an embodiment of this application. Figure 9 As shown, the computer device includes at least one processor 601, a memory 602, at least one network interface 603, and a user interface 605. The various components in the device are coupled together via a bus system 604. It is understood that the bus system 604 is used to implement communication between these components. In addition to a data bus, the bus system 604 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in… Figure 6 The general will label all buses as bus systems.

[0100] The user interface 605 may include a monitor, keyboard, mouse, trackball, clicker, button, touchpad, or touch screen.

[0101] It is understood that memory 602 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM) or programmable read-only memory (PROM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM) and synchronous static random access memory (SSRAM). The memories described in the embodiments of this application are intended to include, but are not limited to, these and any other suitable categories of memory.

[0102] In this embodiment, the memory 602 is used to store various types of data to support the operation of the electronic terminal 600. Examples of this data include any executable program that operates on the electronic terminal 600, such as the operating system 6021 and application programs 6022. The operating system 6021 contains various system programs, such as the framework layer, core library layer, and driver layer, for implementing various basic services and handling hardware-based tasks. The application program 6022 may contain various applications, such as a media player and a browser, for implementing various application services. The medical image style transfer method provided in this embodiment can be included in the application program 6022.

[0103] The methods disclosed in the embodiments of this application can be applied to or implemented by processor 601. Processor 601 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 601 or by instructions in software form. The processor 601 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 601 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. General-purpose processor 601 may be a microprocessor or any conventional processor, etc. The steps of the accessory optimization method provided in the embodiments of this application can be directly reflected as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, which is located in memory. The processor reads the information in the memory and combines it with its hardware to complete the steps of the aforementioned method.

[0104] In an exemplary embodiment, the electronic terminal 600 may be used by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), or complex programmable logic devices (CPLDs) to perform the aforementioned method.

[0105] According to the method provided in the embodiments of this application, this application also provides a computer program product, which includes: computer program code, which, when run on a computer, causes the computer to execute... Figures 1 to 4 The medical image style transfer method of any of the embodiments shown.

[0106] According to the method provided in the embodiments of this application, this application also provides a computer-readable storage medium storing program code, which, when run on a computer, causes the computer to perform the above-described method.

[0107] As used in this specification, the terms "component," "module," "system," etc., are used to refer to computer-related entities, hardware, firmware, combinations of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. As illustrated, applications running on computing devices and computing devices can both be components. One or more components may reside in a process and / or an execution thread, and components may be located on a single computer and / or distributed among two or more computers. Furthermore, these components can be executed from various computer-readable media on which various data structures are stored. Components can communicate, for example, via local and / or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system, and / or a network, such as the Internet interacting with other systems via signals).

[0108] Those skilled in the art will recognize that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0109] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0110] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0111] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0112] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0113] In the above embodiments, the functions of each functional unit can be implemented entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. A computer program product includes one or more computer instructions (programs). When the computer program instructions (programs) are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., high-density digital video discs, DVDs), or semiconductor media (e.g., solid-state disks, SSDs, etc.).

[0114] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0115] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0116] In summary, this application provides a method, system, device, medium, and program product for medical image style transfer. Therefore, the technical solution of this application effectively overcomes the various shortcomings of the prior art and has high industrial applicability.

[0117] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

[0118] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and substitutions can be made without departing from the principle of this technology, and these improvements and substitutions should also be considered within the scope of protection of this application.

Claims

1. A medical image style transfer method, characterized by, include: An image language model is used to perform a first weighted processing on the source medical image and guidance information to obtain source medical image features and guidance information features. The source medical image features and guidance information features are then fused to obtain target condition features. The source medical image is preprocessed to obtain enhanced image features; The target condition features and enhanced image features are fused to obtain fused features; The fusion features are then subjected to a second weighting process to obtain a medical image with the target style.

2. The method according to claim 1, characterized in that, The first weighted processing of the source medical image and guidance information to obtain target conditional features includes: The guidance information is fused with learnable text information to obtain parameter condition features; Feature extraction is performed on the source medical image to obtain reference image style features; The parameter condition features are fused with the style features of the reference image to obtain the target condition features.

3. The method according to claim 1, characterized in that, The target condition features and enhanced image features are fused to obtain the fused features. The fusion employs a cross-attention mechanism; wherein the key matrix and value matrix are derived from enhanced image features, and the query matrix is ​​derived from target condition features.

4. The method according to claim 1, characterized in that, In the process of fusing the target condition features and the enhanced image features to obtain fused features, the enhanced image features are first divided into blocks, and then fused with the target condition features in blocks.

5. The method according to claim 1, characterized in that, The preprocessing includes any one of the following: A random bias field is applied to simulate magnetic field inhomogeneity, random downsampling is performed to simulate different resolutions, and gamma correction, contrast, and brightness adjustments are applied.

6. A model training method, characterized in that, Training an image language model as described in any one of claims 1 to 5 includes: Feature extraction was performed on the source medical image, guidance information, and target style medical image, respectively. The similarity between the source medical image and the target style medical image, and the similarity between the guidance information and the target style medical image are calculated respectively to obtain the image-image similarity matrix and the image-text similarity matrix; Based on the loss between image-image similarity matrix, image-text similarity matrix and standard diagonal matrix, the feature extraction process of source medical image, guidance information and target style medical image is trained respectively.

7. A medical image style transfer system, characterized in that, include: The first weighting module is used to perform a first weighting process on the source medical image and guidance information using an image language model to obtain source medical image features and guidance information features, and to fuse the source medical image features and guidance information features to obtain a first fusion; The preprocessing module is used to preprocess the source medical image to obtain enhanced image features; The second weighting module is used to fuse the target condition features and the enhanced image features to obtain fused features; The style transfer module is used to perform a second weighting process on the fused features to obtain a medical image with a target style.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 5 or 6.

9. A computer program product, characterized in that, The computer program product includes computer program code that, when run on a computer, causes the computer to perform the method of any one of claims 1 to 5 or 6.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the method of any one of claims 1 to 5 or 6.

Images