Medical image registration method, system and device and storage medium
By decomposing medical images into texture and structural parts and fusing the deformation field using an adaptive weighting method, the problem of limited fusion gain in the deformation field in the prior art is solved, and a more accurate medical image registration effect is achieved.
Patent Information
- Application Number
- CN202411738506.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-29
- Publication Date
- 2025-04-15
AI Technical Summary
Existing medical image registration methods extract different scale features through deep neural networks, resulting in a large amount of redundant information between the deformation fields, limited fusion gain, making it difficult to accurately characterize the real deformation field.
By decomposing the fixed image and moving image into texture and structural parts, the deformation field generation model is input respectively to obtain the initial deformation field, texture deformation field and structural deformation field, and the adaptive weighting method is used to fuse these deformation fields to generate a fusion deformation field.
Through the complementary amount of information, the real deformation field can be portrayed more accurately, reducing the computational complexity and improving the registration effect.
Smart Images
Figure CN119832031A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of image processing, and in particular relates to a medical image registration method, system, device and storage medium. Background Art
[0002] Most of the current medical image registration methods adopt a pyramid structure, which uses a deep neural network to extract features of different scales of the image pairs to be registered (fixed image-moving image pairs), calculates the deformation fields at different scales through features of different scales, and then obtains the final deformation field by fusing the deformation fields of different scales layer by layer. Finally, the registered image is obtained through spatial transformation of the moving image and the deformation field.
[0003] However, deep neural networks are used to extract features of different scales of the images to be registered, and then the deformation fields at different scales are calculated using features of different scales. Since the deformation fields at different scales are obtained from features extracted from the same image, there may be a large amount of redundant information between the deformation fields, and the gain brought by the fusion of deformation fields is limited. Summary of the invention
[0004] The purpose of the present invention is to provide a medical image registration method, system, device and storage medium, so that the information contained in the initial deformation field, texture deformation field and structural deformation field can complement each other and more accurately depict the real deformation field.
[0005] A first aspect of the present invention provides a medical image registration method, comprising:
[0006] Acquire an image pair to be registered, wherein the image pair to be registered includes: a fixed image and a moving image;
[0007] Decomposing the fixed image to obtain a fixed texture image and a fixed structure image, and decomposing the moving image to obtain a moving texture image and a moving structure image;
[0008] Inputting the fixed image and the moving image into a preset deformation field generation model to obtain an initial deformation field, inputting the fixed texture image and the moving texture image into the deformation field generation model to obtain a texture deformation field, and inputting the fixed structure image and the moving structure image into the deformation field generation model to obtain a structure deformation field;
[0009] Using a preset weight estimation network to fuse the initial deformation field, the texture deformation field and the structure deformation field to obtain a fused deformation field;
[0010] The registered image is obtained through spatial transformation of the moving image and the fused deformation field.
[0011] In some embodiments, the deformation field generation model includes:
[0012] A feature extraction network is used to extract features from the fixed image, the moving image, the fixed texture image, the moving texture image, the fixed structure image and the moving structure image to obtain corresponding features;
[0013] A splicing layer, used for splicing the features of the fixed image and the features of the moving image to obtain initial features, splicing the features of the fixed texture image and the features of the moving texture image to obtain texture features, and splicing the features of the fixed structure image and the features of the moving structure image to obtain structure features;
[0014] The deformation field estimation network is used to generate an initial deformation field according to the initial features, generate a texture deformation field according to the texture features, and generate a structural deformation field according to the structural features.
[0015] In some embodiments, the training method of the deformation field generation model includes:
[0016] The deformation field generation model and the weight estimation network constitute a calibration network, and the training method of the calibration network includes:
[0017] Acquire a training set, wherein the training set includes a plurality of sample image pairs to be registered;
[0018] Setting a loss function required for neural network training, wherein the loss function includes: a consistency loss function and a smoothness loss function;
[0019] The neural network is iteratively trained according to the training set and the loss function until a convergence condition is reached to obtain a calibrated network.
[0020] In some embodiments, the consistency loss function is:
[0021]
[0022] in, represents the i-th sample moving image after the initial deformation field transformation, represents the i-th sample moving texture image after the texture deformation field transformation, represents the i-th sample moving structure image after the structural deformation field transformation, represents the similarity between the i-th sample moving image and the i-th sample fixed image after the initial deformation field transformation, It represents the similarity between the i-th sample moving texture image and the i-th sample fixed texture image after the texture deformation field transformation. It represents the similarity between the i-th sample moving structure image and the i-th sample fixed structure image after the structural deformation field transformation;
[0023] The smoothness loss function is:
[0024]
[0025] in, Indicates the smoothness of the texture deformation field, L smooth (φ i ) represents the smoothness of the fused deformation field, Indicates the smoothness of the structural deformation field.
[0026] In some embodiments, the iterative training of the neural network according to the training set and the loss function until a convergence condition is reached to obtain a calibrated network includes:
[0027] The neural network is iteratively trained by minimizing the consistency loss function and the smoothness loss function, and the neural network parameters are calculated;
[0028] Substituting the neural network parameters into the neural network obtains a calibrated network.
[0029] In some embodiments, the using of a preset weight estimation network to fuse the initial deformation field, the texture deformation field and the structure deformation field to obtain a fused deformation field includes:
[0030] Obtaining fusion weights through the weight estimation network;
[0031] The calculation formula of the fusion deformation field obtained by the fusion weight calculation is:
[0032] φ=w1*φ T +w2*φ S +w3*φ I
[0033] w3=1-w1-w2
[0034] Wherein, φ represents the fused deformation field, φ T represents the texture deformation field, φ S represents the structural deformation field, φ I represents the initial deformation field, w1 represents the first fusion weight, w2 represents the second fusion weight, and w3 represents the third fusion weight;
[0035] The initial deformation field, the texture deformation field and the structure deformation field are fused by the fused deformation field calculation formula to obtain a fused deformation field.
[0036] A second aspect of the present invention provides a medical image registration system, comprising:
[0037] An image acquisition module, used to acquire a pair of images to be registered, wherein the pair of images to be registered includes: a fixed image and a moving image;
[0038] An image decomposition module, used for decomposing the fixed image to obtain a fixed texture image and a fixed structure image, and decomposing the moving image to obtain a moving texture image and a moving structure image;
[0039] A deformation field generation module, used for inputting the fixed image and the moving image into a preset deformation field generation model to obtain an initial deformation field, inputting the fixed texture image and the moving texture image into the deformation field generation model to obtain a texture deformation field, and inputting the fixed structure image and the moving structure image into the deformation field generation model to obtain a structure deformation field;
[0040] A deformation field fusion module, used to fuse the initial deformation field, the texture deformation field and the structure deformation field using a preset weight estimation network to obtain a fused deformation field;
[0041] The image registration module is used to obtain a registered image through spatial transformation of the moving image and the fused deformation field.
[0042] In some embodiments, the deformation field fusion module includes:
[0043] A weight generating unit, used for obtaining a fusion weight through the weight estimation network;
[0044] The deformation field calculation unit is used to calculate the fusion deformation field by the fusion weight calculation formula:
[0045] φ=w1*φ T +w2*φ S +w3*φ I
[0046] w3=1-w1-w2
[0047] Wherein, φ represents the fused deformation field, φ T represents the texture deformation field, φ S represents the structural deformation field, φ I represents the initial deformation field, w1 represents the first fusion weight, w2 represents the second fusion weight, and w3 represents the third fusion weight;
[0048] The fusion unit is used to fuse the initial deformation field, the texture deformation field and the structure deformation field through the fused deformation field calculation formula to obtain a fused deformation field.
[0049] A third aspect of the present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.
[0050] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.
[0051] The technical solution provided by the present invention has the following advantages and effects: firstly, the fixed image and the moving image are decomposed into texture and structure parts respectively by means of image decomposition; secondly, the texture and structure parts are respectively used to obtain the texture deformation field and the structure deformation field through the deformation field generation model; at the same time, the original image pair is used to obtain the initial deformation field through the deformation field generation model; the initial deformation field, the texture deformation field and the structure deformation field are fused by an adaptive weighted method to obtain a fused deformation field, so that the information contained in the initial deformation field, the texture deformation field and the structure deformation field can complement each other, and the real deformation field can be more accurately portrayed. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] Figure 1 It is a schematic diagram of the process of the medical image registration method provided by the present invention;
[0053] Figure 2 is a structural block diagram of the medical image registration system provided by the present invention;
[0054] Figure 3 It is a diagram of the internal structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0055] In order to facilitate the understanding of the present invention, specific embodiments of the present invention will be described in more detail below with reference to the accompanying drawings.
[0056] Unless otherwise specified or defined, the "first, second..." used in this article is merely used to distinguish names and does not represent a specific quantity or order.
[0057] Unless specifically stated or defined otherwise, the term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.
[0058] It should be noted that, in this document, “fixed to” or “connected to” may mean directly fixing or connecting to an element, or indirectly fixing or connecting to an element.
[0059] like Figure 1 As shown, this embodiment provides a medical image registration method, including the following steps S1 to S5:
[0060] Step S1, obtaining an image pair to be registered, wherein the image pair to be registered includes: a fixed image and a moving image.
[0061] In practical applications, the fixed image and the moving image can be images of the object under different conditions, such as the fixed image is an image of the object taken at a first time, and the moving image is an image of the object of the same type as the fixed image taken at a second time, or the fixed image is an MRI image of the object taken at a first time, and the moving image is a CT image of the object taken at a first time. The image pairs to be registered are registered by a subsequent registration method, thereby achieving information complementarity and better diagnosis.
[0062] Step S2: decomposing the fixed image to obtain a fixed texture image and a fixed structure image, and decomposing the moving image to obtain a moving texture image and a moving structure image.
[0063] In practical applications, image texture information usually represents the geometric shape, outline, edge and corner of the object, etc., which is used to provide the basic framework and layout of the image. Image structure information usually represents color, shadow, granularity and pattern, etc., which is used to provide clues to the surface characteristics of the image. Wavelet transform can be performed on fixed images and moving images to capture the local frequency and spatial information of the image, decompose the image into components of different scales, and thus extract the texture information of the image to obtain fixed texture images and moving texture images. Alternatively, the fixed image and the moving image can be decomposed by feature extraction based on local binary patterns, gray-level co-occurrence matrix and other texture decomposition methods to obtain corresponding fixed texture images and moving texture images. The singular value decomposition method can be used to decompose the fixed image and the moving image to obtain corresponding fixed structure images and moving structure images. Deep learning methods, such as convolutional neural networks, can also be used to decompose fixed images to obtain fixed texture images and fixed structure images, and to decompose moving images to obtain moving texture images and moving structure images.
[0064] Step S3, input the fixed image and the moving image into a preset deformation field generation model to obtain an initial deformation field, input the fixed texture image and the moving texture image into the deformation field generation model to obtain a texture deformation field, and input the fixed structure image and the moving structure image into the deformation field generation model to obtain a structure deformation field.
[0065] In practical applications, the deformation field generation model is a pre-trained model, and the fixed image is represented by I f , the moving image is represented by I m , the fixed texture image is represented as T f , the moving texture image is represented as T m , the fixed structure image is represented as S f , the moving structure image is represented as S mThe fixed texture image and the moving texture image constitute a texture image pair, and the fixed structure image and the moving structure image constitute a structure image pair. The deformation field generation model can generate an initial deformation field according to the image pair to be registered, generate a texture deformation field according to the texture image pair, and generate a structure deformation field according to the structure deformation field, thereby obtaining deformation field information of different dimensions. After the image pair to be registered is decomposed through content, the texture deformation field and the structure deformation field can be obtained from the two perspectives of texture and structure. Compared with the previous multi-scale deformation field, it can be more informative, and the texture deformation field and the structure deformation field can complement each other. In addition, compared with the multi-scale deformation field, the present application does not need to repeatedly calculate the deformation fields of different scales, which reduces the computational complexity.
[0066] Specifically, the deformation field generation model includes:
[0067] A feature extraction network is used to extract features from the fixed image, the moving image, the fixed texture image, the moving texture image, the fixed structure image and the moving structure image to obtain corresponding features;
[0068] A splicing layer, used for splicing the features of the fixed image and the features of the moving image to obtain initial features, splicing the features of the fixed texture image and the features of the moving texture image to obtain texture features, and splicing the features of the fixed structure image and the features of the moving structure image to obtain structure features;
[0069] The deformation field estimation network is used to generate an initial deformation field according to the initial features, generate a texture deformation field according to the texture features, and generate a structural deformation field according to the structural features.
[0070] In practical applications, the feature extraction network can use Resnet, VGG and other network architectures, such as Resnet50. After the image pair to be registered, the texture image pair and the structure image pair are input into the deformation field generation model. The initial image pair, texture image pair and structure image pair are input into the feature extraction network FeaExt to obtain the corresponding features:
[0071]
[0072] For example, the texture image T decomposed from a fixed image f The features extracted by the feature extraction network are
[0073] The feature extraction network extracts features from the fixed image in the image pair to be registered to obtain initial fixed features, and extracts features from the moving image to obtain initial moving features. The initial fixed features and the initial moving features are spliced together through the splicing layer to obtain initial features. The deformation field estimation network can be composed of a convolution layer, an activation layer, and a normalization layer. The initial deformation field corresponding to the initial features is estimated through the deformation field estimation network. The fixed texture image is feature extracted through the feature extraction network to obtain fixed texture features, and the moving texture image is feature extracted to obtain moving texture features. The fixed texture features and the moving texture features are then spliced together through the splicing layer to obtain texture features. The texture deformation field corresponding to the texture features is estimated through the deformation field estimation network. The fixed structure image is feature extracted to obtain fixed structure features, and the moving structure image is feature extracted to obtain moving structure features. The fixed structure features and the moving structure features are then spliced together through the splicing layer to obtain structure features. The structural deformation field corresponding to the structure features is estimated through the deformation field estimation network. The initial deformation field can be expressed as The texture deformation field can be expressed as The structural deformation field can be expressed as
[0074] Specifically, in the present application, the same feature extraction network is used to extract features from the image pairs, texture image pairs and structural image pairs to be registered, and the same deformation field estimation network is used to generate the corresponding deformation fields for the initial features, texture features and structural features, which greatly reduces the number of parameters.
[0075] In practical applications, the deformation field generation model and the weight estimation network constitute a calibration network, and the training method of the calibration network includes:
[0076] Acquire a training set, wherein the training set includes a plurality of sample image pairs to be registered;
[0077] Setting a loss function required for neural network training, wherein the loss function includes: a consistency loss function and a smoothness loss function;
[0078] The neural network is iteratively trained according to the training set and the loss function until a convergence condition is reached to obtain a calibrated network.
[0079] Specifically, the multiple sample image pairs to be registered are multiple image pairs to be registered at different angles and different objects, and the multiple sample image pairs constitute a training set, which is expressed as N represents the number of sample image pairs, represents the sample fixed image in the i-th sample image pair, Represents the sample moving image in the i-th sample image pair. The neural network is iteratively trained through the training set and the loss function until the set convergence condition is reached. In the process of training the neural network, the parameters of the feature extraction network, the deformation field estimation network and the weight estimation network are constrained by the consistency loss function and the smoothness loss function. The consistency loss function is used to constrain the moving image, the moving texture image and the moving structure image to be consistent with the fixed image, the fixed texture image and the fixed structure image after the corresponding deformation field conversion. The correlation loss function, such as the structural similarity function, can be used. The smoothness loss function is used to constrain the deformation field to remain smooth, that is, the deformation field of adjacent pixels needs to be consistent. The total variation loss function can be used to reduce the noise and artifacts in the fused image and enhance the smoothness of the image.
[0080] Specifically, the consistency loss function is:
[0081]
[0082] in, represents the i-th sample moving image after the initial deformation field transformation, represents the i-th sample moving texture image after the texture deformation field transformation, represents the i-th sample moving structure image after the structural deformation field transformation, represents the similarity between the i-th sample moving image and the i-th sample fixed image after the initial deformation field transformation, It represents the similarity between the i-th sample moving texture image and the i-th sample fixed texture image after the texture deformation field transformation. It represents the similarity between the i-th sample moving structure image and the i-th sample fixed structure image after the structural deformation field transformation;
[0083] The smoothness loss function is:
[0084]
[0085] in, Indicates the smoothness of the texture deformation field, L smooth (φ i ) represents the smoothness of the fused deformation field, Indicates the smoothness of the structural deformation field.
[0086] In practical applications, represents the sample fixed texture image decomposed from the i-th sample fixed image, represents the sample moving texture image decomposed from the i-th sample moving image, represents the sample fixed structure image decomposed from the i-th sample fixed image, represents the sample moving structure image decomposed from the i-th sample moving image, represents the i-th sample moving image after the initial deformation field transformation, represents the i-th sample moving texture image after the texture deformation field transformation, represents the i-th sample moving structure image after the structural deformation field transformation, and These three similarities can be defined by cosine similarity. In the process of training the neural network, these three similarities need to be as large as possible so that the moving image, moving texture image and moving structure image are consistent with the fixed image, fixed texture image and fixed structure image after being transformed by the corresponding deformation field.
[0087] Specifically, the iterative training of the neural network according to the training set and the loss function until a convergence condition is reached to obtain a calibrated network includes:
[0088] The neural network is iteratively trained by minimizing the consistency loss function and the smoothness loss function, and the neural network parameters are calculated;
[0089] Substituting the neural network parameters into the neural network obtains a calibrated network.
[0090] In practical applications, after training is completed, the image pairs to be registered, texture image pairs, and structure image pairs are input into the calibration network to obtain the fused deformation field.
[0091] Step S4: using a preset weight estimation network to fuse the initial deformation field, the texture deformation field and the structure deformation field to obtain a fused deformation field.
[0092] Specifically, the weight estimation network includes a convolution layer, a normalization layer, an activation layer and a fully connected layer. The weight estimation network can also be composed of a combination of a channel attention network and a spatial attention network. The fusion of the initial deformation field, the texture deformation field and the structural deformation field is fused using an adaptive weighted method to obtain a more accurate fused deformation field.
[0093] Specifically, the initial deformation field, the texture deformation field and the structure deformation field are fused by using a preset weight estimation network to obtain a fused deformation field, including:
[0094] Obtaining fusion weights through the weight estimation network;
[0095] The calculation formula of the fusion deformation field obtained by the fusion weight calculation is:
[0096] φ=w1*φ T +w2*φ S +w3*φ I
[0097] w3=1-w1-w2
[0098] Wherein, φ represents the fused deformation field, φ T represents the texture deformation field, φ S represents the structural deformation field, φ I represents the initial deformation field, w1 represents the first fusion weight, w2 represents the second fusion weight, and w3 represents the third fusion weight;
[0099] The initial deformation field, the texture deformation field and the structure deformation field are fused by the fused deformation field calculation formula to obtain a fused deformation field.
[0100] In practical applications, the initial deformation field, texture deformation field and structural deformation field are input into the weight estimation network to obtain the first fusion weight and the second fusion weight, and then the third fusion weight is calculated based on the first fusion weight and the second fusion weight. According to the first fusion weight, the second fusion weight and the third fusion weight, the texture deformation field, the structural deformation field and the initial deformation field are weightedly calculated to obtain a fused deformation field, so that the information contained in the initial deformation field, the texture deformation field and the structural deformation field are complementary, and the real deformation field can be more accurately portrayed.
[0101] Step S5: obtaining a registered image by spatially transforming the moving image and the fused deformation field.
[0102] In practical applications, a spatial transformation network can be used to realize the spatial conversion between the moving graphics and the fused deformation field to obtain the registered image. The spatial transformation network includes a grid generator and a sampler. The fused deformation field is used as the spatial transformation parameter. The grid generator constructs a sampling grid according to the spatial transformation parameter to obtain the mapping relationship before and after the spatial transformation. The sampler uses interpolation to generate a feature map after the spatial transformation, that is, to obtain the registered image.
[0103] The medical image registration method of the present invention first decomposes the fixed image and the moving image into texture and structure parts respectively by means of image decomposition. Secondly, the texture and structure parts respectively obtain the texture deformation field and the structure deformation field through the deformation field generation model. At the same time, the original image pair obtains the initial deformation field through the deformation field generation model. The initial deformation field, texture deformation field and structure deformation field are fused by an adaptive weighted method to obtain a fused deformation field, so that the information contained in the initial deformation field, texture deformation field and structure deformation field can complement each other, and can more accurately portray the real deformation field.
[0104] like Figure 2 As shown, an embodiment of the present invention further provides a medical image registration system, comprising:
[0105] The image acquisition module 10 is used to acquire an image pair to be registered, wherein the image pair to be registered includes: a fixed image and a moving image;
[0106] An image decomposition module 20, configured to decompose the fixed image to obtain a fixed texture image and a fixed structure image, and decompose the moving image to obtain a moving texture image and a moving structure image;
[0107] A deformation field generation module 30, used for inputting the fixed image and the moving image into a preset deformation field generation model to obtain an initial deformation field, inputting the fixed texture image and the moving texture image into the deformation field generation model to obtain a texture deformation field, and inputting the fixed structure image and the moving structure image into the deformation field generation model to obtain a structure deformation field;
[0108] A deformation field fusion module 40 is used to fuse the initial deformation field, the texture deformation field and the structure deformation field using a preset weight estimation network to obtain a fused deformation field;
[0109] The image registration module 50 is used to obtain a registered image through spatial transformation of the moving image and the fused deformation field.
[0110] As an optional implementation, the deformation field generation model includes:
[0111] A feature extraction network is used to extract features from the fixed image, the moving image, the fixed texture image, the moving texture image, the fixed structure image and the moving structure image to obtain corresponding features;
[0112] A splicing layer, used for splicing the features of the fixed image and the features of the moving image to obtain initial features, splicing the features of the fixed texture image and the features of the moving texture image to obtain texture features, and splicing the features of the fixed structure image and the features of the moving structure image to obtain structure features;
[0113] The deformation field estimation network is used to generate an initial deformation field according to the initial features, generate a texture deformation field according to the texture features, and generate a structural deformation field according to the structural features.
[0114] As an optional implementation, the deformation field generation model and the weight estimation network constitute a calibration network, and the medical image registration system further includes a training module, which includes the following units not shown in the figure:
[0115] A training set acquisition unit, used to acquire a training set, wherein the training set includes a plurality of sample image pairs to be registered;
[0116] A function setting unit, used to set a loss function required for neural network training, wherein the loss function includes: a consistency loss function and a smoothness loss function;
[0117] The training unit is used to iteratively train the neural network according to the training set and the loss function until a convergence condition is reached to obtain a calibrated network.
[0118] As an optional implementation, the consistency loss function is:
[0119]
[0120] in, represents the i-th sample moving image after the initial deformation field transformation, represents the i-th sample moving texture image after the texture deformation field transformation, represents the i-th sample moving structure image after the structural deformation field transformation, represents the similarity between the i-th sample moving image and the i-th sample fixed image after the initial deformation field transformation, It represents the similarity between the i-th sample moving texture image and the i-th sample fixed texture image after the texture deformation field transformation. It represents the similarity between the i-th sample moving structure image and the i-th sample fixed structure image after the structural deformation field transformation;
[0121] The smoothness loss function is:
[0122]
[0123] in, Indicates the smoothness of the texture deformation field, L smooth (φ i ) represents the smoothness of the fused deformation field, Indicates the smoothness of the structural deformation field.
[0124] As an optional implementation, the training unit includes the following units not shown in the figure:
[0125] The iterative training of the neural network according to the training set and the loss function until a convergence condition is reached to obtain a calibrated network comprises:
[0126] The neural network is iteratively trained by minimizing the consistency loss function and the smoothness loss function, and the neural network parameters are calculated;
[0127] Substituting the neural network parameters into the neural network obtains a calibrated network.
[0128] As an optional implementation, the image registration module includes the following units not shown in the figure:
[0129] A weight generating unit, used for obtaining a fusion weight through the weight estimation network;
[0130] The deformation field calculation unit is used to calculate the fusion deformation field by the fusion weight calculation formula:
[0131] φ=w1*φ T +w2*φ S +w3*φ I
[0132] w3=1-w1-w2
[0133] Wherein, φ represents the fused deformation field, φ T represents the texture deformation field, φ S represents the structural deformation field, φ I represents the initial deformation field, w1 represents the first fusion weight, w2 represents the second fusion weight, and w3 represents the third fusion weight;
[0134] The fusion unit is used to fuse the initial deformation field, the texture deformation field and the structure deformation field through the fused deformation field calculation formula to obtain a fused deformation field.
[0135] Each module of the medical image registration system can be implemented in whole or in part by software, hardware, or a combination thereof. Each module and unit can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.
[0136] like Figure 3 As shown, an embodiment of the present invention discloses a computer device, including a memory and a processor, wherein the memory stores a computer program;
[0137] The computer device may be a server, and its internal structure diagram may be as follows: Figure 3 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the medical image registration method described in the above embodiments is implemented.
[0138] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0139] An embodiment of the present invention further discloses a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the medical image registration method described in the above embodiments.
[0140] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
[0141] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
Claims
1. A medical image registration method, characterized in that: The following steps are involved: Acquire an image pair to be registered, wherein the image pair to be registered includes: a fixed image and a moving image; Decomposing the fixed image to obtain a fixed texture image and a fixed structure image, and decomposing the moving image to obtain a moving texture image and a moving structure image; Inputting the fixed image and the moving image into a preset deformation field generation model to obtain an initial deformation field, inputting the fixed texture image and the moving texture image into the deformation field generation model to obtain a texture deformation field, and inputting the fixed structure image and the moving structure image into the deformation field generation model to obtain a structure deformation field; Using a preset weight estimation network to fuse the initial deformation field, the texture deformation field and the structure deformation field to obtain a fused deformation field; The registered image is obtained through spatial transformation of the moving image and the fused deformation field.
2. The medical image registration method according to claim 1, characterized in that: The deformation field generation model includes: A feature extraction network is used to extract features from the fixed image, the moving image, the fixed texture image, the moving texture image, the fixed structure image and the moving structure image to obtain corresponding features; A splicing layer, used for splicing the features of the fixed image and the features of the moving image to obtain initial features, splicing the features of the fixed texture image and the features of the moving texture image to obtain texture features, and splicing the features of the fixed structure image and the features of the moving structure image to obtain structure features; The deformation field estimation network is used to generate an initial deformation field according to the initial features, generate a texture deformation field according to the texture features, and generate a structural deformation field according to the structural features.
3. The medical image registration method according to claim 1, characterized in that: The deformation field generation model and the weight estimation network constitute a calibration network, and the training method of the calibration network includes: Acquire a training set, wherein the training set includes a plurality of sample image pairs to be registered; Setting a loss function required for neural network training, wherein the loss function includes: a consistency loss function and a smoothness loss function; The neural network is iteratively trained according to the training set and the loss function until a convergence condition is reached to obtain a calibrated network.
4. The medical image registration method according to claim 3, characterized in that: The consistency loss function is: in, represents the i-th sample moving image after the initial deformation field transformation, represents the i-th sample moving texture image after the texture deformation field transformation, represents the i-th sample moving structure image after the structural deformation field transformation, represents the similarity between the i-th sample moving image and the i-th sample fixed image after the initial deformation field transformation, It represents the similarity between the i-th sample moving texture image and the i-th sample fixed texture image after the texture deformation field transformation. It represents the similarity between the i-th sample moving structure image and the i-th sample fixed structure image after the structural deformation field transformation; The smoothness loss function is: in, Indicates the smoothness of the texture deformation field, L smooth (φ i ) represents the smoothness of the fused deformation field, Indicates the smoothness of the structural deformation field.
5. The medical image registration method according to claim 4, characterized in that: The iterative training of the neural network according to the training set and the loss function until a convergence condition is reached to obtain a calibrated network comprises: The neural network is iteratively trained by minimizing the consistency loss function and the smoothness loss function, and the neural network parameters are calculated; Substituting the neural network parameters into the neural network obtains a calibrated network.
6. The medical image registration method according to claim 1, characterized in that: The method of using a preset weight estimation network to fuse the initial deformation field, the texture deformation field and the structure deformation field to obtain a fused deformation field includes: Obtaining fusion weights through the weight estimation network; The calculation formula of the fusion deformation field obtained by the fusion weight calculation is: φ=w1*φ T +w2*φ S +w3*φ I w3=1-w1-w2 Wherein, φ represents the fused deformation field, φ T represents the texture deformation field, φ S represents the structural deformation field, φ I represents the initial deformation field, w1 represents the first fusion weight, w2 represents the second fusion weight, and w3 represents the third fusion weight; The initial deformation field, the texture deformation field and the structure deformation field are fused by the fused deformation field calculation formula to obtain a fused deformation field.
7. A medical image registration system, characterized in that: include: An image acquisition module, used to acquire a pair of images to be registered, wherein the pair of images to be registered includes: a fixed image and a moving image; An image decomposition module, used for decomposing the fixed image to obtain a fixed texture image and a fixed structure image, and decomposing the moving image to obtain a moving texture image and a moving structure image; A deformation field generation module, used for inputting the fixed image and the moving image into a preset deformation field generation model to obtain an initial deformation field, inputting the fixed texture image and the moving texture image into the deformation field generation model to obtain a texture deformation field, and inputting the fixed structure image and the moving structure image into the deformation field generation model to obtain a structure deformation field; A deformation field fusion module, used to fuse the initial deformation field, the texture deformation field and the structure deformation field using a preset weight estimation network to obtain a fused deformation field; The image registration module is used to obtain a registered image through spatial transformation of the moving image and the fused deformation field.
8. The medical image registration system according to claim 7, characterized in that: The deformation field fusion module comprises: A weight generating unit, used for obtaining a fusion weight through the weight estimation network; The deformation field calculation unit is used to calculate the fusion deformation field by the fusion weight calculation formula: φ=w1*φ T +w2*φ S +w3*φ I w3=1-w1-w2 Wherein, φ represents the fused deformation field, φ T represents the texture deformation field, φ S represents the structural deformation field, φ I represents the initial deformation field, w1 represents the first fusion weight, w2 represents the second fusion weight, and w3 represents the third fusion weight; The fusion unit is used to fuse the initial deformation field, the texture deformation field and the structure deformation field through the fused deformation field calculation formula to obtain a fused deformation field.
9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.