Medical image registration method and system based on language large model prompt

Abstract

This invention belongs to the field of image registration technology, specifically relating to a medical image registration method and system based on a large language model prompt. The method includes: acquiring multi-regional medical images and corresponding medical image description text; inputting the medical images and corresponding medical image description text into a multimodal adaptive feature encoder to extract medical image features and medical image description features; fusing the medical image features with given target medical image features to generate hybrid features, predicting an initial deformation field based on the hybrid features, performing multi-level deformation optimization on the initial deformation field to generate an optimized deformation field, and obtaining a medical image registration model; based on the multi-regional medical images, employing a dynamic data balancing strategy to amplify the medical image data of different regions, and inputting the amplified multi-regional medical images into the medical image registration model to obtain multi-regional medical image registration results.

Description

Technical Field

[0001] This invention belongs to the field of image registration technology, specifically relating to a medical image registration method and system based on language large model prompts. Background Technology

[0002] Medical image registration technology is a core component of medical image analysis. It aims to align medical images from different times or modalities through spatial transformation, so that anatomical structures are consistent in spatial position, providing a reliable basis for clinical diagnosis and treatment, and assisting doctors in accurately identifying lesions, planning treatment plans, and evaluating efficacy.

[0003] The human anatomy is complex and diverse, with significant morphological differences among organs, requiring registration algorithms to be highly adaptable. Furthermore, medical images are susceptible to interference from various factors during acquisition, such as imaging equipment noise and artifacts caused by voluntary or involuntary patient movements, increasing the difficulty of registration. In addition, the registration process needs to maintain the topological integrity of the anatomical structures, avoiding structural breaks, distortions, and other abnormalities that could affect analytical accuracy. However, existing registration methods often struggle to achieve precise alignment.

[0004] Currently, deep learning methods are widely used to improve the alignment accuracy of registration methods, but many limitations remain. On the one hand, deep learning models have not fully integrated prior medical knowledge, resulting in a lack of effective guidance. On the other hand, existing models are insufficient in their modeling capabilities when faced with complex anatomical deformations. Moreover, because different anatomical sites have different requirements for registration accuracy, existing methods struggle to balance the accuracy of each site, easily leading to local registration biases. These problems severely restrict the effectiveness of medical image registration in clinical applications. Summary of the Invention

[0005] This invention provides a medical image registration method and system based on a large language model prompt. By deeply coupling medical images and their descriptive text, it achieves adaptive extraction of image features guided by the descriptive text. Multi-level deformation optimization maintains the spatial continuity of the deformation field, and a dynamic data balancing strategy is used to amplify and train the medical image data, addressing the class imbalance problem and ensuring model stability. This enhances the model's ability to model different anatomical structures and achieves high-precision spatial alignment of multimodal medical images while maintaining sub-voxel-level registration accuracy.

[0006] A medical image registration method based on language large model prompts includes:

[0007] Acquire medical images from multiple regions and their corresponding medical image description text;

[0008] Based on medical images and corresponding medical image description text, they are input into a multimodal adaptive feature encoder to extract medical image features and medical image description features;

[0009] The medical image features are fused with the features of a given target medical image to generate a hybrid feature. The initial deformation field is predicted based on the hybrid feature. The initial deformation field is then optimized through multi-level deformation to generate an optimized deformation field, thus obtaining a medical image registration model.

[0010] Based on multi-region medical images, a dynamic data balancing strategy is used to amplify medical image data from different regions. The amplified multi-region medical images are then input into a medical image registration model to obtain multi-region medical image registration results.

[0011] By deeply coupling medical images and their descriptive text, adaptive extraction of image features guided by the descriptive text is achieved. The spatial continuity of the deformation field is maintained through multi-level deformation optimization, and the class imbalance problem in medical image data is solved by amplifying the training of medical image data through a dynamic data balancing strategy, ensuring the stability of the model. This improves the model's ability to model different anatomical structures and achieves high-precision spatial alignment of multimodal medical images while maintaining sub-voxel level registration accuracy.

[0012] Furthermore, based on the medical image and its corresponding medical image description text, these are input into a multimodal adaptive feature encoder to extract medical image features and medical image description features, including:

[0013] Based on the medical image description text, it is input into the CLIP text encoder for text encoding to obtain the medical image description features;

[0014] Based on the descriptive features of medical images, they are input into a multilayer perceptron (MLP) to generate dynamic weight parameters and bias terms for a three-dimensional convolutional network.

[0015] Based on medical images, feature encoding is performed using 3D convolution operations to generate medical image features.

[0016] By guiding the adaptive feature extraction of medical images with the extracted medical image descriptive features, the network can understand and utilize prior knowledge of medical images, significantly improving the discriminative power of feature representation.

[0017] Furthermore, the medical image features are fused with the features of a given target medical image to generate hybrid features. An initial deformation field is predicted based on these hybrid features. The initial deformation field is then subjected to multi-level deformation optimization to generate an optimized deformation field, resulting in a medical image registration model, including:

[0018] Based on medical image features and given target medical image features, a hybrid feature is generated using Hadamard interactive feature fusion operation.

[0019] Based on the hybrid features, a three-level 3×3×3 convolutional network is used to process and predict the initial displacement field, which is then used as the initial deformation field.

[0020] Based on the initial deformation field, the displacement field is optimized using the integral flow field algorithm to generate an optimized deformation field, thus obtaining a medical image registration model.

[0021] By employing a progressive deformation optimization strategy that first generates an initial deformation field and then generates an optimized deformation field, common folding or tearing phenomena can be effectively avoided.

[0022] Furthermore, when using the integral flow field algorithm to optimize the displacement field, a pyramid architecture is used to perform progressive sampling of each resolution layer from fine to fine, and the displacement field is corrected by combining the medical image features of the current resolution layer to obtain the optimized displacement field.

[0023] By adopting a step-by-step optimization strategy from coarse to fine, global consistency in registration can be guaranteed, and subtle local deformations can also be restored.

[0024] Furthermore, based on multi-region medical images, a dynamic data balancing strategy is used to augment the medical image data of different regions. The augmented multi-region medical images are then input into a medical image registration model to obtain multi-region medical image registration results, including:

[0025] Based on multi-region medical images, obtain the number of samples of medical images in each region;

[0026] Based on the number of medical images in each region, the largest number of samples in each region is selected as the data sample.

[0027] Based on the number of medical image samples and data samples in each region, the amplification factor of medical images in each region is calculated.

[0028] Based on the calculated amplification factor of the medical images in each region, the small sample data is subjected to cyclical repeated loading for amplification processing until the effective sample number of the medical images in each region reaches a balance.

[0029] Based on the amplified medical images of each region, they are input into the medical image registration model to obtain the regional medical image registration results.

[0030] Furthermore, based on performing cyclical reloading amplification processing on small sample data until the effective sample number of medical images in each region reaches a balance, it also includes:

[0031] Real-time monitoring of the loss value of medical image data in each region during amplification processing;

[0032] Based on the difference between the loss value of medical image data in each region and the global average loss value, the Sigmoid function is used to achieve a smooth transition, and the adjustment factor is calculated.

[0033] The sample acquisition frequency is dynamically adjusted based on the adjustment factor to balance the number of effective samples of medical images in each region.

[0034] By designing an intelligent dynamic data balancing strategy, the class imbalance problem in medical image data can be effectively solved, enabling the model to achieve stable registration performance in various anatomical locations.

[0035] A system for medical image registration based on language large model prompts includes:

[0036] The image acquisition module is used to acquire medical images of multiple regions and corresponding medical image description text;

[0037] The feature extraction module is used to extract medical image features and medical image description features by inputting the medical image and the corresponding medical image description text into the multimodal adaptive feature encoder.

[0038] The model building module is used to fuse medical image features with given target medical image features to generate hybrid features, predict the initial deformation field based on the hybrid features, perform multi-level deformation optimization on the initial deformation field to generate an optimized deformation field, and obtain a medical image registration model.

[0039] The image registration module is used to augment medical image data from different regions based on multi-region medical images using a dynamic data balancing strategy. The augmented multi-region medical images are then input into the medical image registration model to obtain the multi-region medical image registration results.

[0040] An electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method described above.

[0041] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described above.

[0042] The beneficial effects of this invention are as follows:

[0043] This invention achieves adaptive extraction of image features guided by medical image description text by deeply coupling medical images and medical image descriptive text; it maintains the spatial continuity of the deformation field through multi-level deformation optimization, and solves the class imbalance problem in medical image data by amplifying and training medical image data through a dynamic data balancing strategy, thus ensuring the stability of the model; it can improve the model's ability to model different anatomical structures, and achieve high-precision spatial alignment of multimodal medical images while maintaining sub-voxel level registration accuracy. Attached Figure Description

[0044] Figure 1 This is a flowchart of the present invention;

[0045] Figure 2 A flowchart of a multimodal adaptive feature encoder;

[0046] Figure 3 A flowchart of the deformation field;

[0047] Figure 4 Flowchart for an image registration network;

[0048] Figure 5 A schematic diagram showing the image registration results for brain MRI, cardiac MRI, and abdominal CT.

[0049] Figure 6 This is a schematic diagram of the system structure of the present invention;

[0050] Figure 7 This is a schematic diagram of the structure of a computer device. Detailed Implementation

[0051] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this disclosure, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using other structures and / or functionalities besides one or more of the aspects set forth herein.

[0053] In addition, specific details are provided in the following description to facilitate a thorough understanding of the examples, and those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0054] Example 1

[0055] Figure 1 This paper presents a medical image registration method based on a large language model. By deeply coupling medical images and their descriptive text, it achieves adaptive feature extraction guided by the descriptive text. Multi-level deformation optimization maintains the spatial continuity of the deformation field, and a dynamic data balancing strategy is used to augment and train the medical image data, addressing class imbalance and ensuring model stability. This method enhances the model's ability to model different anatomical structures and achieves high-precision spatial alignment of multimodal medical images while maintaining sub-voxel-level registration accuracy. The specific steps include:

[0056] S1: Obtain medical images from multiple regions and corresponding medical image description text;

[0057] In this embodiment, the medical image is represented as follows: , Indicates different regions of a medical image. That is, brain MRI medical images Cardiac MRI medical images Abdominal CT medical images The medical image description text corresponding to the medical image is represented as follows: .

[0058] S2: Based on the medical image and the corresponding medical image description text, input them into the multimodal adaptive feature encoder to extract medical image features and medical image description features;

[0059] Figure 2The diagram illustrates the extraction of medical image features and medical image descriptive features using a multimodal adaptive feature encoder. It includes:

[0060] S21: Based on the medical image description text, input it into the CLIP text encoder for text encoding to obtain the medical image description features;

[0061] The CLIP text encoder is used for text encoding, resulting in the following expression for the medical image description features:

[0062] ;

[0063] In the formula, Represented as medical image descriptive features; Indicates a text encoder; Represents the space of real numbers; Indicates the feature dimension.

[0064] S22: Based on the medical image description features, input them into the multilayer perceptron (MLP) to generate dynamic weight parameters and bias terms for the three-dimensional convolutional network;

[0065] The expression for the dynamic weight parameters of the 3D convolutional network, generated using a multilayer perceptron (MLP), is as follows:

[0066] ;

[0067] In the formula, Represents the dynamic weight parameters of a 3D convolutional network; This represents a multilayer perceptron; Indicates the number of output channels; Indicates the number of input channels; Indicates the kernel size;

[0068] The expression for the generated bias term is:

[0069] ;

[0070] In the formula, Indicates the bias term; This represents a multilayer perceptron;

[0071] S23: Based on medical images, feature encoding is performed using three-dimensional convolution operations to generate medical image features;

[0072] The expression for generating medical image features using 3D convolution operations for feature encoding is as follows:

[0073] ;

[0074] In the formula, Representing medical image features; Represents the ReLU activation function; Represents a 3D convolution operation; Indicates the number of channels; Indicates altitude; Indicates width; Indicates depth;

[0075] S3: The medical image features are fused with the given target medical image features to generate hybrid features, and the initial deformation field is predicted based on the hybrid features. The initial deformation field is then optimized through multi-level deformation to generate an optimized deformation field, thus obtaining a medical image registration model.

[0076] Figure 3 The diagram shown illustrates the generation of the optimized deformation field. It includes:

[0077] S31: Based on medical image features and given target medical image features, Hadamard interactive feature fusion operation is used to generate hybrid features;

[0078] In this embodiment, the expression for Hadamard interactive feature fusion operation is:

[0079] ;

[0080] In the formula, Indicates mixed features, i.e. ; Represents the features of the source medical image, namely ; Represents the features of a given target medical image, i.e. ;

[0081] S32: Based on the hybrid features, a three-level 3×3×3 convolutional network is used for processing to predict the initial displacement field, which is used as the initial deformation field.

[0082] The initial displacement field is predicted using a three-level 3×3×3 convolutional network, and its expression is as follows:

[0083] ;

[0084] In the formula, Represents the initial displacement field; Represents a 3D convolution operation; Indicates the activation function;

[0085] S33: Based on the initial deformation field, the displacement field is optimized using the integral flow field algorithm to generate the optimized deformation field and obtain the medical image registration model;

[0086] In the displacement field optimization using the integral flow field algorithm, a pyramid architecture is employed to perform progressive sampling from fine to fine at each resolution layer. Specifically, at each resolution layer, triple linear interpolation is used to upsample the displacement field of the previous resolution layer, and the displacement field is corrected by incorporating the medical image features of the current resolution layer to obtain the optimized displacement. This progressive optimization strategy ensures global consistency in registration and also recovers subtle local deformations.

[0087] The expression for the correction process is as follows:

[0088] ;

[0089] In the formula, Indicates the first Displacement field of each resolution layer; Indicates the first Displacement field of each resolution layer; Indicates an upsampling operation; Indicates the first The correction amount for each resolution layer.

[0090] In this embodiment, the resolution hierarchy consists of five layers, including... , , , and the original resolution.

[0091] The displacement field is optimized using an integral flow field algorithm to generate an optimized deformation field, the expression of which is:

[0092] ;

[0093] In the formula, This represents the displacement field obtained after optimization by the integral flow field algorithm, which refers to the desired target state, facilitating subsequent related operations or analysis. This represents the learning rate parameter, used to update the step size; Indicates the integration step size; Indicates from the initial time to The intermediate displacement field state generated during the time process.

[0094] Figure 4 The diagram shows the entire image registration network process. DW represents the displacement field deformation module.

[0095] S4: Based on multi-regional medical images, a dynamic data balancing strategy is used to amplify the medical image data of different regions, and the amplified multi-regional medical images are input into the medical image registration model to obtain the multi-regional medical image registration results.

[0096] S41: Based on multi-region medical images, obtain the number of samples of medical images in each region;

[0097] In this embodiment, the number of brain MRI medical images is set to be [number]. The number of samples of cardiac MRI medical images is The number of samples of abdominal CT medical images is .

[0098] S42: Based on the number of medical images in each region, select the largest number of samples among the medical images in each region as the data sample;

[0099] The expression for the data sample is as follows:

[0100] ;

[0101] In the formula, This represents the maximum number of samples among the medical images in each region. Represents the maximum value function;

[0102] S43: Calculate the amplification factor of medical images in each region based on the number of medical image samples and data samples in each region;

[0103] The expression for the amplification factor of medical images in each region is as follows:

[0104] ;

[0105] In the formula, Indicates the magnification factor of medical images in each region; This represents the rounding operation function;

[0106] S44: Based on the calculated amplification factor of the medical images in each region, perform cyclical reloading amplification processing on the small sample data until the effective sample number of the medical images in each region reaches a balance.

[0107] This includes amplification processing based on cyclically loading small sample data until the number of effective samples in each region of the medical image reaches a balance, and also includes:

[0108] S441: Real-time monitoring of the loss value of medical image data in each region during the amplification process;

[0109] S442: Based on the difference between the loss value of medical image data in each region and the global average loss value, the Sigmoid function is used to achieve a smooth transition, and the adjustment factor is calculated.

[0110] The expression for the adjustment factor is:

[0111] ;

[0112] In the formula, Indicates the regulating factor; Indicates the first Loss values ​​of medical images in each region; This represents the global average loss value; Represents the Sigmoid function;

[0113] S443: The sample acquisition frequency is dynamically adjusted according to the adjustment factor to balance the number of effective samples of medical images in each region.

[0114] S45: Based on the amplified medical images of each region, input them into the medical image registration model to obtain the regional medical image registration results.

[0115] In this embodiment, Figure 5 The diagram shown illustrates the registration results of brain MRI, cardiac MRI, and abdominal CT scans. It can be seen that:

[0116] (1) Based on the brain MRI registration results, the model is highly consistent with the fine structures such as the ventricles and sulci, which verifies the model's adaptability to complex brain anatomy.

[0117] (2) Based on the cardiac MRI registration results, the deformation of the myocardial systolic-diastolic cycle was accurately tracked in dynamic cardiac images, especially in the estimation of ventricular wall motion. The registered time series images were superior to traditional methods in terms of motion artifact suppression and phase alignment.

[0118] (3) Based on the abdominal CT registration results, even under nonlinear deformation caused by respiratory motion, the model can still maintain topological integrity and no folding or tearing occurs.

[0119] Based on the above, the registration results fully demonstrate the superiority of the proposed universal medical image registration model based on language large model prompts on multi-site and multi-modal data. Through text-guided feature encoding and adaptive deformation field prediction, the model can accurately capture the anatomical characteristics of different organs, achieving high-precision registration and providing a reliable spatial alignment tool for clinical diagnosis and surgical planning.

[0120] Example 2

[0121] Based on the same technological concept, such as Figure 6 As shown, this embodiment also provides a medical image registration system based on a large language model prompt, including an image acquisition module, a feature extraction module, a model construction module, and an image registration module.

[0122] Specifically, the image acquisition module is used to acquire medical images of multiple regions and corresponding medical image description text;

[0123] Specifically, the feature extraction module is used to extract medical image features and medical image description features by inputting the medical image and the corresponding medical image description text into the multimodal adaptive feature encoder.

[0124] Specifically, the model building module is used to fuse medical image features with given target medical image features to generate hybrid features, predict the initial deformation field based on the hybrid features, perform multi-level deformation optimization on the initial deformation field to generate an optimized deformation field, and obtain a medical image registration model.

[0125] Specifically, the image registration module is used to augment medical image data from different regions based on multi-region medical images using a dynamic data balancing strategy, and then inputs the augmented multi-region medical images into the medical image registration model to obtain multi-region medical image registration results.

[0126] Example 3

[0127] Based on the same technical concept, embodiments of this application also provide a computer device, including a memory 1 and a processor 2, such as... Figure 7 As shown, the memory 1 stores a computer program, and the processor 2 executes the computer program to implement any of the methods described above.

[0128] The memory 1 includes at least one type of readable storage medium, such as flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 1 can be an internal storage unit of the language-based large model prompting medical image registration system, such as a hard disk. In other embodiments, the memory 1 can also be an external storage device of the language-based large model prompting medical image registration system, such as a plug-in hard disk, a smart media card (SMC), a secure digital card (SD), a flash card, etc. Furthermore, the memory 1 can include both internal storage units and external storage devices of the language-based large model prompting medical image registration system. The memory 1 can be used not only to store application software and various types of data installed in the language-based large model prompting medical image registration system, such as the code of the language-based large model prompting medical image registration system program, but also to temporarily store data that has been output or will be output.

[0129] In some embodiments, processor 2 may be a central processing unit (CPU), graphics processing unit (GPU), controller, microcontroller, microprocessor, or other data processing chip, used to run program code stored in memory 1 or process data, such as executing a medical image registration system program based on language large model prompts.

[0130] The present invention also discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps described in the above-described method embodiments. The storage medium may be a volatile or non-volatile computer-readable storage medium.

[0131] The computer program product of the application page content refresh method provided in the embodiments of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the steps of the method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0132] The present invention also discloses a computer program that, when executed by a processor, implements any of the methods described in the foregoing embodiments. This computer program product can be implemented specifically through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0133] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0134] It should be noted that in the description of this invention, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this invention, unless otherwise stated, "a plurality of" means at least two.

[0135] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0136] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0137] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0138] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0139] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.

[0140] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0141] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A medical image registration method based on language large model prompts, characterized in that, include: Acquire medical images from multiple regions and their corresponding medical image description text; Based on medical images and corresponding medical image description text, they are input into a multimodal adaptive feature encoder to extract medical image features and medical image description features; The medical image features are fused with the features of a given target medical image to generate a hybrid feature. The initial deformation field is predicted based on the hybrid feature. The initial deformation field is then optimized through multi-level deformation to generate an optimized deformation field, thus obtaining a medical image registration model. Based on multi-regional medical images, a dynamic data balancing strategy is used to amplify medical image data from different regions, and the amplified multi-regional medical images are input into a medical image registration model to obtain multi-regional medical image registration results. The medical image registration model is obtained by fusing features from medical images with features from a given target medical image to generate hybrid features, predicting an initial deformation field based on the hybrid features, and then performing multi-level deformation optimization on the initial deformation field to generate an optimized deformation field. Based on medical image features and given target medical image features, a hybrid feature is generated using Hadamard interactive feature fusion operation. Based on the hybrid features, a three-level 3×3×3 convolutional network is used to process and predict the initial displacement field, which is then used as the initial deformation field. Based on the initial deformation field, the displacement field is optimized using the integral flow field algorithm to generate the optimized deformation field and obtain the medical image registration model. When using the integral flow field algorithm to optimize the displacement field, a pyramid architecture is used to sample each resolution layer from fine to fine, and the displacement field is corrected by combining the medical image features of the current resolution layer to obtain the optimized displacement field.

2. The medical image registration method based on a large language model as described in claim 1, characterized in that, Based on medical images and corresponding medical image description text, these are input into a multimodal adaptive feature encoder to extract medical image features and medical image description features, including: Based on the medical image description text, it is input into the CLIP text encoder for text encoding to obtain the medical image description features; Based on the descriptive features of medical images, they are input into a multilayer perceptron (MLP) to generate dynamic weight parameters and bias terms for a three-dimensional convolutional network. Based on medical images, feature encoding is performed using 3D convolution operations to generate medical image features.

3. The medical image registration method based on a large language model as described in claim 1, characterized in that, Based on multi-region medical images, a dynamic data balancing strategy is used to augment medical image data from different regions. The augmented multi-region medical images are then input into a medical image registration model to obtain multi-region medical image registration results, including: Based on multi-region medical images, obtain the number of samples of medical images in each region; Based on the number of medical images in each region, the largest number of samples in each region is selected as the data sample. Based on the number of medical image samples and data samples in each region, the amplification factor of medical images in each region is calculated. Based on the calculated amplification factor of the medical images in each region, the small sample data is subjected to cyclical repeated loading for amplification processing until the effective sample number of the medical images in each region reaches a balance. Based on the amplified medical images of each region, they are input into the medical image registration model to obtain the regional medical image registration results.

4. The medical image registration method based on a large language model as described in claim 3, characterized in that, Amplification processing based on repeated loading of small sample data until the effective sample count of medical images in each region reaches a balance also includes: Real-time monitoring of the loss value of medical image data in each region during amplification processing; Based on the difference between the loss value of medical image data in each region and the global average loss value, and using the Sigmoid function to achieve a smooth transition, the adjustment factor is calculated; The sample acquisition frequency is dynamically adjusted based on the adjustment factor to balance the number of effective samples of medical images in each region.

5. A system for medical image registration based on language large model prompts as described in claim 1, characterized in that, include: The image acquisition module is used to acquire medical images of multiple regions and corresponding medical image description text; The feature extraction module is used to extract medical image features and medical image description features by inputting the medical image and the corresponding medical image description text into the multimodal adaptive feature encoder. The model building module is used to fuse medical image features with given target medical image features to generate hybrid features, predict an initial deformation field based on the hybrid features, and perform multi-level deformation optimization on the initial deformation field to generate an optimized deformation field, thus obtaining a medical image registration model. Specifically, based on medical image features and given target medical image features, a Hadamard interactive feature fusion operation is used to generate hybrid features. Based on the hybrid features, a three-level 3×3×3 convolutional network is used to predict an initial displacement field, which serves as the initial deformation field. Based on the initial deformation field, an integral flow field algorithm is used to optimize the displacement field, generating an optimized deformation field, thus obtaining a medical image registration model. When using the integral flow field algorithm to optimize the displacement field, a pyramid architecture is used to sample each resolution layer from fine to fine, and the displacement field is corrected by combining the medical image features of the current resolution layer to obtain an optimized displacement field. The image registration module is used to augment medical image data from different regions based on multi-region medical images using a dynamic data balancing strategy. The augmented multi-region medical images are then input into the medical image registration model to obtain the multi-region medical image registration results.

6. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 4.

Images