Image registration method and device, storage medium and computer equipment
The image is registered through the deformation field model, and the problem of low image registration efficiency in the prior art is solved, real-time registration and rapid subtraction image generation are realized.
Patent Information
- Application Number
- CN202111069450.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2021-09-06
- Filing Date
- 2021-09-13
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2041-09-13
AI Technical Summary
The prior art is difficult to achieve real-time performance and low efficiency in image registration, especially during angiography, the artifacts caused by changes in blood vessels and structural tissues caused by physiological movements of the patient, and it is difficult to effectively reduce artifacts.
Using the deformation field model, by obtaining the first image and the second image to be registered, inputting them into the trained deformation field model, obtaining the target deformation field data, and registering the first image based on this data, obtaining the first image and the second image. The registration image corresponding to the two images. This method includes the calculation of feature difference values and the training of deformation field models, and iterating the model with a loss function to improve registration accuracy.
Real-time image registration is realized, image registration efficiency is improved, subtraction image generation process is simplified, and rapid generation efficiency of video and registration film are improved.
Smart Images

Figure CN113989338B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to an image registration method and device, a storage medium and a computer device. Background Art
[0002] Digital subtraction angiography is a new X-ray imaging system and one of the most commonly used devices for clinical precision treatment and guided interventional surgery. During angiography, the flow state of the contrast agent is observed by subtracting the fluoroscopic image without contrast agent injection from the fluoroscopic image with contrast agent injection to obtain blood flow information in the area.
[0003] In view of the relative position changes of blood vessels and structural tissues caused by physiological movements of patients during angiography, registration and other methods are currently often used to adjust the image displacement, reduce artifacts caused by movement, and then subtract the angiography film and the registered image. The current registration method is: use the edge points of the image to confirm the feature points, and then use the temporal movement law of the feature points to confirm the control nodes that are ultimately used for radiation transformation. Finally, this method uses the acquired nodes for registration and subtraction to obtain a subtraction map. However, this method of using spatiotemporal information for feature point confirmation and registration is difficult to achieve real-time performance, and the efficiency of using this method for image registration is low. Summary of the invention
[0004] In view of this, the present application provides an image registration method and apparatus, a storage medium and a computer device, which help to improve the efficiency of image registration.
[0005] According to one aspect of the present application, there is provided an image registration method, comprising:
[0006] Acquire a first image to be registered and a second image corresponding to the first image;
[0007] Inputting the first image and the second image into a trained deformation field model to obtain target deformation field data;
[0008] Based on the target deformation field data, the first image is registered to obtain a registered image corresponding to the second image.
[0009] Optionally, a feature difference value between the first image and the second image is less than a preset difference threshold.
[0010] Optionally, the method for calculating the feature difference value includes:
[0011] Determine a first image feature corresponding to the first image and a second image feature corresponding to the second image, wherein the first image feature includes any one of a first grayscale feature, a first structure edge feature, and a first anatomical feature of the first image, and the second image feature includes any one of a second grayscale feature, a second structure edge feature, and a second anatomical feature of the second image, and the first image feature and the second image feature are of the same type;
[0012] The feature difference value is calculated based on the first image feature and the second image feature; the preset difference threshold includes any one of a preset grayscale threshold, a preset structure threshold and a preset anatomical threshold, the preset grayscale difference threshold has a value range of [0.01, 0.2], and the unit is a percentage, the preset structure difference threshold is a grayscale threshold, and the value range is [0.01, 0.4], and the unit is a percentage, the preset structure difference threshold is a position threshold, and the value range is [20, 200] pixels, and the preset anatomical threshold has a value range of [5, 100] pixels.
[0013] Optionally, the first image includes a target non-contrast frame image, and the second image includes a target contrast frame image.
[0014] Optionally, the target deformation field data includes a matrix containing displacement information of each pixel point when the target non-contrast frame image is deformed toward the target contrast frame image.
[0015] Optionally, the training method of the deformation field model includes:
[0016] Acquire at least one sample image pair, wherein any sample image pair includes an image to be registered and a standard registration image;
[0017] Inputting the image to be registered and the standard registered image in any sample image pair into the deformation field model to be trained to obtain deformation field data, and registering the image to be registered in any sample image pair according to the deformation field data to obtain a sample registered image corresponding to any sample image pair;
[0018] The deformation field model is iterated using a loss function based on a feature difference value between the standard registration image and the sample registration image corresponding to any one of the sample image pairs.
[0019] Optionally, the iterating the deformation field model by using a loss function based on the feature difference value between the standard registered image and the sample registered image corresponding to any one of the sample image pairs specifically includes:
[0020] Determine a structural segmentation image of a standard registration image in any sample image pair by using a structural extraction model, and downweight corresponding pixels of the sample registration image based on the structural segmentation image;
[0021] The loss function of the deformation field model is determined based on a downweighted perceptual loss function of the feature difference value between the downweighted sample registration image corresponding to any sample image pair and the standard registration image and a regularization term corresponding to the deformation field model, and the deformation field model is iterated using the loss function.
[0022] According to another aspect of the present application, a subtraction image generation method is provided, comprising:
[0023] Acquire a first image to be registered and a second image corresponding to the first image;
[0024] Inputting the first image and the second image into a trained deformation field model to obtain target deformation field data;
[0025] Based on the target deformation field data, registering the first image to obtain a registered image corresponding to the second image;
[0026] Subtract the second image from the registered image to obtain a target subtracted image.
[0027] According to another aspect of the present application, a deformation field model generation device is provided, comprising:
[0028] An image acquisition module to be registered, used to acquire a first image to be registered and a second image corresponding to the first image;
[0029] A target deformation data acquisition module, used for inputting the first image and the second image into a trained deformation field model to acquire target deformation field data;
[0030] The registration module is used to register the first image based on the target deformation field data to obtain a registered image corresponding to the second image.
[0031] Optionally, a feature difference value between the first image and the second image is less than a preset difference threshold.
[0032] Optionally, the device further comprises:
[0033] A difference value calculation module is used to determine a first image feature corresponding to the first image and a second image feature corresponding to the second image, wherein the first image feature includes any one of a first grayscale feature, a first structural edge feature and a first anatomical feature of the first image, and the second image feature includes any one of a second grayscale feature, a second structural edge feature and a second anatomical feature of the second image, and the first image feature and the second image feature are of the same type; the feature difference value is calculated based on the first image feature and the second image feature; the preset grayscale difference threshold has a value range of [0.01, 0.2], and the unit is percentage; the preset structural difference threshold is a grayscale threshold, and the value range is [0.01, 0.4], and the unit is percentage; the preset structural difference threshold is a position threshold, and the value range is [20, 200] pixels; the value range of the preset anatomical threshold is [5, 100] pixels.
[0034] Optionally, the first image includes a target non-contrast frame image, and the second image includes a target contrast frame image.
[0035] Optionally, the target deformation field data includes a matrix containing displacement information of each pixel point when the target non-contrast frame image is deformed toward the target contrast frame image.
[0036] Optionally, the device further comprises:
[0037] A model training module is used to obtain at least one sample image pair, wherein any sample image pair includes an image to be registered and a standard registration image; the image to be registered and the standard registration image in any sample image pair are input into the deformation field model to be trained to obtain deformation field data, and the image to be registered in any sample image pair is registered according to the deformation field data to obtain a sample registration image corresponding to any sample image pair; based on the feature difference value between the standard registration image and the sample registration image corresponding to any sample image pair, the deformation field model is iterated using a loss function.
[0038] Optionally, the model training module is also used to use the structure extraction model to determine the structure segmentation image of the standard registration image in any sample image pair, and downweight the corresponding pixels of the sample registration image based on the structure segmentation image; determine the loss function of the deformation field model based on the downweighted perceptual loss function of the feature difference value between the downweighted sample registration image corresponding to any sample image pair and the standard registration image and the regularization term corresponding to the deformation field model, and iterate the deformation field model using the loss function.
[0039] According to another aspect of the present application, a deformation field model generation device is provided, comprising:
[0040] An image acquisition module to be registered, used to acquire a first image to be registered and a second image corresponding to the first image;
[0041] A target deformation data acquisition module, used for inputting the first image and the second image into a trained deformation field model to acquire target deformation field data;
[0042] A registration module, configured to register the first image based on target deformation field data to obtain a registered image corresponding to the second image;
[0043] A subtraction module is used to subtract the second image from the registered image to obtain a target subtracted image.
[0044] According to another aspect of the present application, a storage medium is provided, on which a computer program is stored, and the program implements the above method when executed by a processor.
[0045] According to another aspect of the present application, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor implements the above method when executing the program.
[0046] By means of the above technical solution, the present application provides an image registration method and device, storage medium and computer equipment, the method comprising: obtaining a first image to be registered and a second image corresponding to the first image; inputting the first image and the second image into a trained deformation field model to obtain target deformation field data; based on the target deformation field data, registering the first image to obtain a registered image corresponding to the second image. Compared with the registration method based on spatiotemporal information in the prior art, the present application can realize real-time registration of images, improve the efficiency of image registration, and further help to quickly generate subtraction images using registration films and angiography films, which helps to increase the efficiency of generating subtraction images.
[0047] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0049] Figure 1A schematic diagram of a process of an image registration method provided in an embodiment of the present application is shown;
[0050] Figure 2 A schematic diagram of the workflow of a deformation field model provided in an embodiment of the present application is shown;
[0051] Figure 3 A schematic structural diagram of an image registration device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION
[0052] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.
[0053] In this embodiment, an image registration method is provided, such as Figure 1 As shown, the method includes:
[0054] Step S1, acquiring a first image to be registered and a second image corresponding to the first image;
[0055] Step S2, inputting the first image and the second image into a trained deformation field model to obtain target deformation field data;
[0056] Step S3: registering the first image based on the target deformation field data to obtain a registered image corresponding to the second image.
[0057] In the above embodiment, a first image to be registered and a second image corresponding to the first image are obtained, wherein the method provided in this embodiment is used to realize deformation of the first image into the second image. Specifically, the first image and the second image are input into a trained deformation field model to obtain target deformation field data, which is data for deformation of the first image into the second image. In actual application scenarios, the first image and the second image may be images of the same size, and the target deformation field data includes displacement information of each pixel in the first image. The first image is registered using the displacement information to obtain a registered image corresponding to the second image.
[0058] Optionally, the first image includes a target non-contrast frame image, and the second image includes a target contrast frame image; and a feature difference value between the first image and the second image is less than a preset difference threshold.
[0059] In this embodiment, when performing angiography on a patient, the image is collected by a digital subtraction angiography (DSA) device. Usually, during angiography, the contrast agent is injected after a delay of several frames. The image without contrast agent is used as a mask, i.e., a non-contrast frame image, and the frame image containing contrast agent is used as an angiography film, i.e., an angiography frame image. If the feature difference value between the first image and the second image is less than the preset difference threshold, it means that the degree of motion artifact is small. In this case, the mask can be first registered so that the edge of the tissue structure corresponding to the mask is close to the angiography film, and then the subtraction is performed. Avoid the problem that the way of registering the mask may cause the tissue structure image to be unnatural due to the large degree of displacement of the tissue structure.
[0060] In addition, optionally, the target deformation field data includes a matrix containing displacement information of each pixel point when the target non-contrast frame image is deformed toward the target contrast frame image. The deformation field model can specifically select a neural network model. For example, a neural network such as a densely connected convolutional network can be selected as a deformation field model. The first image and the second image are input into the model. Using the network, the image pair can be processed through the network to generate deformation field data. The deformation field data is a matrix containing displacement information of each pixel point when the first image is deformed toward the second image. The first image is registered using the obtained deformation field data so that the first image is deformed into a registered image.
[0061] Compared with the registration method based on spatiotemporal information in the prior art, the embodiment of the present application can realize real-time registration of images, improve the efficiency of image registration, and further help to quickly generate subtraction images using registration films and angiography films, which helps to improve the efficiency of generating subtraction images.
[0062] In an embodiment of the present application, optionally, the method for calculating the feature difference value includes: determining a first image feature corresponding to the first image and a second image feature corresponding to the second image, wherein the first image feature includes any one of a first grayscale feature, a first structural edge feature, and a first anatomical feature of the first image, and the second image feature includes any one of a second grayscale feature, a second structural edge feature, and a second anatomical feature of the second image, and the first image feature and the second image feature are of the same type; calculating the feature difference value based on the first image feature and the second image feature; the preset difference threshold includes any one of a preset grayscale threshold, a preset structural threshold, and a preset anatomical threshold, and the preset grayscale difference threshold has a value range of [0.01, 0.2], and the unit is percentage, that is, the grayscale difference threshold is the difference between the grayscale mean of the first image and the grayscale mean of the second image. The preset structural difference threshold is a grayscale threshold with a value range of [0.01, 0.4], and the unit is percentage, that is, the structural difference threshold is the ratio of the difference between the grayscale mean of the edge feature of the first image and the grayscale mean of the edge feature of the second image to the grayscale mean of the edge feature of the first image, or the preset structural difference threshold is a position threshold with a value range of [20, 200] pixels, that is, the structural difference threshold is the distance between the pixel position of the edge feature of the first image and the pixel position of the edge feature of the second image. The deformation field of the first image and the second image can be calculated by the optical flow method or the like, and the deformation degree of the image is calculated by using the displacement of the deformation field. The preset structural difference threshold is a position threshold with a value range of [20, 200] pixels, and the preset anatomical threshold has a value range of [5, 100] pixels.
[0063] In this embodiment, the image features of the mask and the angiogram are calculated respectively, including but not limited to the grayscale features of the image, the structural edge features, etc. The grayscale features are specifically the grayscale value features of each point in the image, the structural edge features specifically include the position features of the anatomical structure edges contained in the image, and the anatomical structure features specifically include the high-dimensional features obtained by a specific neural network, which can be obtained by training an X-ray image containing any human anatomical information as a training sample. The neural network can be obtained by training including but not limited to performing tasks such as segmentation, noise reduction, reconstruction, and automatic encoders, and performing parameter iteration and optimization. Then, the feature difference between the mask and the angiogram is calculated to determine the degree of motion artifacts, wherein the feature difference is reflected by the grayscale feature difference value and the structural feature difference value, the grayscale feature difference value can specifically be the grayscale mean square error between each pixel point of the mask and the angiogram, and the structural feature difference value can specifically be the difference of the structural edge feature, etc.
[0064] In this embodiment, a deformation field model generation method is also provided, the method comprising:
[0065] Step 101, obtaining at least one sample image pair, wherein any sample image pair includes an image to be registered and a standard registration image;
[0066] Step 102, inputting the image to be registered and the standard registered image in any sample image pair into the deformation field model to be trained to obtain deformation field data, and registering the image to be registered in any sample image pair according to the deformation field data to obtain a sample registered image corresponding to any sample image pair;
[0067] Step 103: based on the feature difference value between the standard registration image and the sample registration image corresponding to any one of the sample image pairs, iterate the deformation field model using a loss function.
[0068] In an embodiment of the present application, first, a sample image pair is obtained, wherein the sample image in the sample image pair (i.e., the image to be registered and the standard registered image) can be an image collected by a digital subtraction angiography (DSA) device when performing angiography on the sample object. Usually, during angiography, the contrast agent is injected with a delay of several frames, and the image without contrast agent is used as a mask, i.e., a non-contrast frame image, and the frame image containing contrast agent is used as an angiography film, i.e., an angiography frame image. In this embodiment, the sample image may include a sample non-contrast frame image and a sample angiography frame image, or may only include a sample angiography frame image.
[0069] Optionally, step 101 may specifically include:
[0070] Step 101-1, obtaining a sample image, wherein the sample image includes at least one sample non-contrast frame image and at least one sample contrast frame image;
[0071] Step 101 - 2, pairing any sample non-contrast frame image with any sample contrast frame image to obtain a plurality of candidate image pairs;
[0072] Step 101-3, calculate the feature difference value between the two images contained in each candidate image pair, and screen out the candidate image pairs whose feature difference values are greater than the preset sample threshold as the sample image pairs, wherein the sample non-contrast frame image in any sample image pair is used as the image to be registered, and the sample contrast frame image is used as the standard registration image.
[0073] In the above embodiment, the sample non-contrast frame image and the sample contrast frame image of the sample object can be first collected by the DSA device as sample images, and the obtained sample images can be further paired to form candidate image pairs. In order to ensure the model training effect and enable the trained deformation field model to effectively align the image motion artifacts, the paired candidate image pairs can be screened. For any candidate image pair, if the feature difference value between the two images contained in the candidate image pair is greater than the preset sample threshold, then the candidate image pair can be used as a sample image pair for subsequent model training, thereby ensuring effective training of the model. Among them, the sample image pair can be composed of a sample non-contrast frame image and a sample contrast frame image, or it can be composed of two sample contrast frame images, as long as the feature difference value between the two images in the sample image pair finally used for training is greater than the above threshold. For the sample image pair composed of two contrast frame images, any one of them can be used as the image to be registered, and the other can be used as the standard registration image. For a sample image pair consisting of a contrast frame image and a non-contrast frame image, the non-contrast frame image is used as the image to be registered, and the contrast frame image is used as the standard registration image. In order to improve the pairing efficiency, in this embodiment, one can be directly selected from multiple sample non-contrast frame images and one can be directly selected from multiple sample contrast frame images to form a candidate image pair.
[0074] Secondly, if Figure 2 As shown, a deformation field model to be trained is constructed. Specifically, a neural network model can be selected. For example, a neural network such as a densely connected convolutional network can be selected as a deformation field model. The image to be registered and the standard registration image in the sample image pair are input into the deformation field model. Using the network, the image pair can be processed by the network to generate deformation field data. The deformation field data is a matrix containing the displacement information of each pixel point that deforms the image to be registered to the standard registration image. The image to be registered is registered with the obtained deformation field data, so that the image to be registered is deformed into a sample registration map.
[0075] Finally, the characteristic difference value between the sample registration image and the standard registration image is calculated, and the loss function of the deformation field model is determined based on the characteristic difference value. The deformation field model is iterated using the loss function until the loss function of the deformation field model is stabilized within a certain range, and the training is completed. When the model is used subsequently, only one mask and one angiogram need to be input to obtain the deformation field data of the mask, thereby realizing the registration of the mask. There is no need to analyze multiple angiograms, and the real-time registration of the mask can be realized, which improves the efficiency of image registration, and then helps to quickly generate subtraction images using registration images and angiograms, and also improves the efficiency of generating subtraction images.
[0076] By applying the technical solution of this embodiment, a sample image pair is obtained, the image pair includes an image to be registered and a standard registration image, and the feature difference value between the two is greater than a preset sample threshold value. Further, the sample image pair is input into the deformation field model to be trained to obtain deformation field data, thereby using the deformation field data to register the image to be registered in the sample image pair, generating a sample registration image, and finally using the feature difference value between the sample registration image and the standard registration image in the sample image pair to determine the loss function of the model, and iterating the model to complete the training of the model. Compared with the current registration method based on spatiotemporal information, the embodiment of the present application can realize real-time registration of masks, improve the efficiency of image registration, and then help to quickly generate subtraction images using registration films and angiography films, and also improve the efficiency of generating subtraction images.
[0077] In the embodiment of the present application, optionally, in step 101-3, "calculating the feature difference value between the two images included in each candidate image pair" specifically includes:
[0078] Step 101-3-1, determining a first sample feature corresponding to the image to be registered and a second sample feature corresponding to the standard registration image, wherein the first sample feature includes any one of a first sample grayscale feature, a first sample structure edge feature, and a first sample anatomical feature of the image to be registered, and the second sample feature includes any one of a second sample grayscale feature, a second sample structure edge feature, and a second sample anatomical feature of the standard registration image, and the first sample feature and the second sample feature are of the same type;
[0079] Step 101-3-2, calculate the sample grayscale feature difference value between the image to be registered and the standard registration image based on the first sample grayscale feature and the second sample grayscale feature, or calculate the sample structural feature difference value between the image to be registered and the standard registration image based on the first structural edge feature and the second structural edge feature, or calculate the sample anatomical feature difference value between the image to be registered and the standard registration image based on the first anatomical feature and the second anatomical feature, wherein the feature difference value includes any one of the grayscale feature difference value, the structural feature difference value, and the anatomical feature difference value.
[0080] In this embodiment, the image features of the mask and the angiogram are calculated respectively, including but not limited to the grayscale features of the image, the structural edge features, etc. The grayscale features are specifically the grayscale value features of each point in the image, the structural edge features specifically include the position features of the anatomical structure edges contained in the image, and the anatomical structure features specifically include the high-dimensional features obtained by a specific neural network, which can be obtained by training an X-ray image containing any human anatomical information as a training sample. The neural network can be obtained by training including but not limited to performing tasks such as segmentation, noise reduction, reconstruction, and automatic encoders, and performing parameter iteration and optimization. Then, the feature difference between the mask and the angiogram is calculated to determine the degree of motion artifacts, wherein the feature difference is reflected by the grayscale feature difference value and the structural feature difference value, the grayscale feature difference value can specifically be the grayscale mean square error between each pixel point of the mask and the angiogram, and the structural feature difference value can specifically be the difference of the structural edge feature, etc.
[0081] In the embodiment of the present application, optionally, step 103 of “determining the loss function of the deformation field model according to the feature difference value between the standard registration image and the sample registration image corresponding to any sample image pair” may specifically include:
[0082] Step 103 - 1 : taking the feature difference value between the image to be registered and the standard registered image corresponding to any sample image pair as the loss function of the deformation field model.
[0083] In the above embodiment, an optional method for determining the loss function is to directly use the feature difference value between the sample registration image and the standard registration image as the registration loss function. In a specific application scenario, the method for calculating the feature difference value between the two can refer to step 101-3-1 and step 101-3-2, which will not be repeated here.
[0084] In the embodiment of the present application, optionally, step 103 of “determining the loss function of the deformation field model according to the feature difference value between the standard registration image and the sample registration image corresponding to any sample image pair” may specifically include:
[0085] Step 103-2, using the structure extraction model, determine the structure segmentation image of the standard registration image in any sample image pair, and downweight the corresponding pixels of the sample registration image based on the structure segmentation image; use the feature difference value between the downweighted sample registration image corresponding to any sample image pair and the standard registration image as the downweighted perceptual loss function of the deformation field model; determine the loss function of the deformation field model based on the downweighted perceptual loss function and the regularization term corresponding to the deformation field model.
[0086] In the above embodiment, in order to solve the problem of unnatural graphics caused by registration, and the problem that blood vessels and other regions are difficult to register due to the fact that there are no blood vessels and other tissue structures in the mask but there are blood vessels and other tissue structures in the angiography, resulting in different image information (this problem can also be understood as the tissue structure in the angiography is blocked by the structure after the angiography, the image information is lost, and this part cannot be matched through image information). The loss function of the deformation field model can be determined by the regularization term + tissue deweighted perception loss. Take the down-weighting of blood vessels as an example, where the down-weighting loss function for blood vessels can be obtained through a pre-trained blood vessel extraction network. The specific steps are: use a pre-trained blood vessel extraction model such as a blood vessel segmentation network to perform blood vessel segmentation on the input standard registration image, and the segmentation result is a tissue segmentation image (here specifically a blood vessel segmentation image); then, use the blood vessel segmentation image to down-weight the sample registration image, specifically, record the pixels with blood vessels in the blood vessel segmentation image, and use these pixel information to down-weight the pixels corresponding to the sample registration image (i.e., multiply by a coefficient of <1), so as to reduce the impact caused by blood vessels; finally, calculate the feature difference value between the down-weighted sample registration image and the standard registration image, and use this value as the down-weighted perceptual loss function of the deformation field model. Furthermore, in order to avoid the occurrence of model overfitting, the down-weighted perceptual loss function is added to the regularization term corresponding to the model, and the sum of the two is used as the loss function of the model. Among them, the expression of the regularization term can be: Among them, L all is the overall loss function, L(x) represents the loss function of calculating the difference between the target and the target, such as perceptual loss, L2 loss, etc., λ is the weight of the L2 regularization term, which represents the proportion of the regularization term in the loss function; x-x' is the gradient value of the image at pixel i and the surrounding pixels. is the L2 regularization penalty term, which calculates the sum of the squares of the gradients of all pixels in the image.
[0087] In an embodiment of the present application, optionally, it also includes: using the remaining sample image pairs to continue to input into the deformation field model to be trained, until the loss function of the obtained deformation field model meets the preset registration condition, thereby obtaining a trained deformation field model.
[0088] In this embodiment, the deformation field model is also trained in the above manner for the remaining sample image pairs, and the training termination condition of the model may be that the loss function of the model satisfies a preset registration condition, for example, the loss function is stable within a preset range.
[0089] In this embodiment, a subtraction image generation method is also provided, including:
[0090] Step 201, acquiring a first image to be registered and a second image corresponding to the first image;
[0091] Step 202, inputting the first image and the second image into a trained deformation field model to obtain target deformation field data;
[0092] Step 203, registering the first image based on the target deformation field data to obtain a registered image corresponding to the second image;
[0093] Step 204: Subtract the second image from the registered image to obtain a target subtracted image.
[0094] In the above embodiment, the deformation field model can be used to implement image registration and subtraction. Specifically, the target non-contrast frame image (i.e., the first image) and the target contrast frame image (i.e., the second image) are obtained and input into the trained deformation field model. The target registration data (i.e., the target deformation field data) of the target non-contrast frame image is obtained through the deformation field model, and the target non-contrast frame image is registered using the target registration data to generate a registered image. Finally, the target contrast frame image and the registered image are subtracted to obtain the final target subtraction image. In this way, a subtraction image can be obtained using one frame of mask and one frame of angiography film without waiting for the acquisition time of multiple frames of angiography film. Image registration and subtraction image generation can be performed quickly, thereby improving image processing efficiency. In this embodiment, steps 201 to 203 are the same as in step 204. Figure 1 Steps S1 to S3 of the embodiment shown are the same and will not be described again. After obtaining the registered image, the second image and the registered image are subtracted to obtain the target subtracted image. In addition, regarding the deformation field model in this embodiment, its training method can refer to the description of steps 101 to 103 above.
[0095] Further, as Figure 1 The specific implementation of the method, the embodiment of the present application provides an image registration device, such as Figure 3 As shown, the device comprises:
[0096] An image acquisition module to be registered, used to acquire a first image to be registered and a second image corresponding to the first image;
[0097] A target deformation data acquisition module, used for inputting the first image and the second image into a trained deformation field model to acquire target deformation field data;
[0098] The registration module is used to register the first image based on the target deformation field data to obtain a registered image corresponding to the second image.
[0099] Optionally, a feature difference value between the first image and the second image is less than a preset difference threshold.
[0100] Optionally, the device further comprises:
[0101] A difference value calculation module is used to determine a first image feature corresponding to the first image and a second image feature corresponding to the second image, wherein the first image feature includes any one of a first grayscale feature, a first structural edge feature and a first anatomical feature of the first image, and the second image feature includes any one of a second grayscale feature, a second structural edge feature and a second anatomical feature of the second image, and the first image feature and the second image feature are of the same type; the feature difference value is calculated based on the first image feature and the second image feature; the preset grayscale difference threshold has a value range of [0.01, 0.2], and the unit is percentage; the preset structural difference threshold is a grayscale threshold, and the value range is [0.01, 0.4], and the unit is percentage; the preset structural difference threshold is a position threshold, and the value range is [20, 200] pixels; the value range of the preset anatomical threshold is [5, 100] pixels.
[0102] Optionally, the first image includes a target non-contrast frame image, and the second image includes a target contrast frame image.
[0103] Optionally, the target deformation field data includes a matrix containing displacement information of each pixel point when the target non-contrast frame image is deformed toward the target contrast frame image.
[0104] Optionally, the device further comprises:
[0105] A model training module is used to obtain at least one sample image pair, wherein any sample image pair includes an image to be registered and a standard registration image; the image to be registered and the standard registration image in any sample image pair are input into the deformation field model to be trained to obtain deformation field data, and the image to be registered in any sample image pair is registered according to the deformation field data to obtain a sample registration image corresponding to any sample image pair; based on the feature difference value between the standard registration image and the sample registration image corresponding to any sample image pair, the deformation field model is iterated using a loss function.
[0106] Optionally, the model training module is also used to use the structure extraction model to determine the structure segmentation image of the standard registration image in any sample image pair, and downweight the corresponding pixels of the sample registration image based on the structure segmentation image; determine the loss function of the deformation field model based on the downweighted perceptual loss function of the feature difference value between the downweighted sample registration image corresponding to any sample image pair and the standard registration image and the regularization term corresponding to the deformation field model, and iterate the deformation field model using the loss function.
[0107] According to another aspect of the present application, a subtraction image generating device is provided, comprising:
[0108] An image acquisition module to be registered, used to acquire a first image to be registered and a second image corresponding to the first image;
[0109] A target deformation data acquisition module, used for inputting the first image and the second image into a trained deformation field model to acquire target deformation field data;
[0110] A registration module, configured to register the first image based on target deformation field data to obtain a registered image corresponding to the second image;
[0111] A subtraction module is used to subtract the second image from the registered image to obtain a target subtracted image.
[0112] It should be noted that for other corresponding descriptions of the functional units involved in the subtraction image generation device provided in the embodiment of the present application, reference can be made to Figure 1 to Figure 2 The corresponding description in the method will not be repeated here.
[0113] Based on the above Figure 1 to Figure 2 The method shown in the embodiment of the present application accordingly provides a storage medium on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned Figure 1 to Figure 2 The method shown.
[0114] Based on this understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each implementation scenario of the present application.
[0115] Based on the above Figure 1 to Figure 2 The method shown in the figure, and the virtual device embodiment, in order to achieve the above purpose, the embodiment of the present application also provides a computer device, which can be a personal computer, a server, a network device, etc., and the computer device includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to achieve the above Figure 1 to Figure 2 The method shown.
[0116] Optionally, the computer device may further include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a WI-FI module, etc. The user interface may include a display, an input unit such as a keyboard, etc., and the optional user interface may also include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (such as a Bluetooth interface, a WI-FI interface), etc.
[0117] Those skilled in the art will appreciate that the computer device structure provided in this embodiment does not limit the computer device, and may include more or fewer components, or a combination of certain components, or different component arrangements.
[0118] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages and saves the hardware and software resources of the computer device, and supports the operation of information processing programs and other software and / or programs. The network communication module is used to realize communication between the components inside the storage medium, and communication with other hardware and software in the physical device.
[0119] Through the description of the above implementation methods, those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform, or by hardware to obtain a first image to be registered and a second image corresponding to the first image; input the first image and the second image into a trained deformation field model to obtain target deformation field data; based on the target deformation field data, register the first image to obtain a registered image corresponding to the second image. Compared with the registration method based on spatiotemporal information in the prior art, the present application can realize real-time registration of images, improve the efficiency of image registration, and further help to quickly generate subtraction images using registration films and angiography films, which helps to increase the efficiency of generating subtraction images.
[0120] Those skilled in the art will appreciate that the accompanying drawings are only schematic diagrams of a preferred implementation scenario, and the modules or processes in the accompanying drawings are not necessarily necessary for implementing the present application. Those skilled in the art will appreciate that the modules in the devices in the implementation scenario can be distributed in the devices of the implementation scenario according to the description of the implementation scenario, or can be changed accordingly and located in one or more devices different from the present implementation scenario. The modules of the above-mentioned implementation scenario can be combined into one module, or can be further split into multiple submodules.
[0121] The above serial numbers of this application are only for description and do not represent the advantages and disadvantages of the implementation scenarios. The above disclosure is only a few specific implementation scenarios of this application, but this application is not limited to them, and any changes that can be thought of by technicians in this field should fall within the scope of protection of this application.
Claims
1. An image registration method, characterized in that: include: Acquire a first image to be registered and a second image corresponding to the first image, wherein a feature difference value between the first image and the second image is less than a preset difference threshold; Inputting the first image and the second image into a trained deformation field model to obtain target deformation field data; Based on the target deformation field data, registering the first image to obtain a registered image corresponding to the second image; The method for calculating the characteristic difference value includes: Determine a first image feature corresponding to the first image and a second image feature corresponding to the second image, wherein the first image feature includes any one of a first grayscale feature, a first structure edge feature, and a first anatomical feature of the first image, and the second image feature includes any one of a second grayscale feature, a second structure edge feature, and a second anatomical feature of the second image, and the first image feature and the second image feature are of the same type; The feature difference value is calculated based on the first image feature and the second image feature; the preset difference threshold includes any one of a preset grayscale difference threshold, a preset structural difference threshold and a preset anatomical threshold, and the preset grayscale difference threshold has a value range of [0.01, 0.2], and the unit is a percentage; the preset structural difference threshold is a grayscale threshold, and the value range is [0.01, 0.4], and the unit is a percentage, or the preset structural difference threshold is a position threshold, and the value range is [20, 200] pixels; the preset anatomical threshold has a value range of [5, 100] pixels.
2. The method according to claim 1, characterized in that The first image includes a target non-contrast frame image, the second image includes a target contrast frame image, and the target deformation field data includes a matrix containing displacement information of each pixel point for deforming the target non-contrast frame image to the target contrast frame image.
3. The method according to claim 1, characterized in that The training method of the deformation field model includes: Acquire at least one sample image pair, wherein any sample image pair includes an image to be registered and a standard registration image; Inputting the image to be registered and the standard registered image in any sample image pair into the deformation field model to be trained to obtain deformation field data, and registering the image to be registered in any sample image pair according to the deformation field data to obtain a sample registered image corresponding to any sample image pair; The deformation field model is iterated using a loss function based on a feature difference value between the standard registration image and the sample registration image corresponding to any one of the sample image pairs.
4. The method according to claim 3, characterized in that The step of iterating the deformation field model by using a loss function based on the feature difference value between the standard registration image and the sample registration image corresponding to any one of the sample image pairs specifically includes: Determine a structural segmentation image of a standard registration image in any sample image pair by using a structural extraction model, and downweight corresponding pixels of the sample registration image based on the structural segmentation image; The loss function of the deformation field model is determined based on a downweighted perceptual loss function of the feature difference value between the downweighted sample registration image corresponding to any sample image pair and the standard registration image and a regularization term corresponding to the deformation field model, and the deformation field model is iterated using the loss function.
5. A subtraction image generation method, characterized in that: include: Acquire a first image to be registered and a second image corresponding to the first image, wherein a feature difference value between the first image and the second image is less than a preset difference threshold; Inputting the first image and the second image into a trained deformation field model to obtain target deformation field data; Based on the target deformation field data, registering the first image to obtain a registered image corresponding to the second image; Subtracting the second image from the registered image to obtain a target subtracted image; The method for calculating the characteristic difference value includes: Determine a first image feature corresponding to the first image and a second image feature corresponding to the second image, wherein the first image feature includes any one of a first grayscale feature, a first structure edge feature, and a first anatomical feature of the first image, and the second image feature includes any one of a second grayscale feature, a second structure edge feature, and a second anatomical feature of the second image, and the first image feature and the second image feature are of the same type; The feature difference value is calculated based on the first image feature and the second image feature; the preset difference threshold includes any one of a preset grayscale difference threshold, a preset structural difference threshold and a preset anatomical threshold, and the preset grayscale difference threshold has a value range of [0.01, 0.2], and the unit is a percentage; the preset structural difference threshold is a grayscale threshold, and the value range is [0.01, 0.4], and the unit is a percentage, or the preset structural difference threshold is a position threshold, and the value range is [20, 200] pixels; the preset anatomical threshold has a value range of [5, 100] pixels.
6. An image registration device, characterized in that: include: An image acquisition module to be registered, used to acquire a first image to be registered and a second image corresponding to the first image, wherein a feature difference value between the first image and the second image is less than a preset difference threshold; A target deformation data acquisition module, used for inputting the first image and the second image into a trained deformation field model to acquire target deformation field data; A registration module, configured to register the first image based on target deformation field data to obtain a registered image corresponding to the second image; a difference value calculation module, configured to determine a first image feature corresponding to the first image and a second image feature corresponding to the second image, wherein the first image feature includes any one of a first grayscale feature, a first structure edge feature, and a first anatomical feature of the first image, and the second image feature includes any one of a second grayscale feature, a second structure edge feature, and a second anatomical feature of the second image, and the first image feature and the second image feature are of the same type; and calculate the feature difference value according to the first image feature and the second image feature; The preset difference threshold includes any one of a preset grayscale difference threshold, a preset structural difference threshold and a preset anatomical threshold. The preset grayscale difference threshold has a value range of [0.01, 0.2], and the unit is percentage; the preset structural difference threshold is a grayscale threshold, and the value range is [0.01, 0.4], and the unit is percentage, or the preset structural difference threshold is a position threshold, and the value range is [20, 200] pixels; the preset anatomical threshold has a value range of [5, 100] pixels.
7. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.
8. A computer device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 4 is implemented.
