Subtraction image generation method, device, storage medium and computer equipment

Abstract

The present application discloses a subtraction image generation method, device, storage medium and computer equipment, the method comprising: obtaining a target non-contrast frame image and a target contrast frame image corresponding to a target object, and calculating an image difference value; if the image difference value is less than a preset difference threshold, registering the target non-contrast frame to obtain a target registered image, and subtracting the target contrast frame image and the target registered image to obtain a first target subtraction image of the target object; if the image difference value is greater than or equal to the preset difference threshold, inputting the target contrast frame image into a trained subtraction model to obtain a second target subtraction image of the target object.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a subtraction image generation method, device, storage medium and computer equipment. Background Art

[0002] Digital subtraction angiography is a new X-ray imaging system and one of the most commonly used devices for clinical precision treatment and guided interventional surgery. During angiography, the flow state of the contrast agent is observed by subtracting the fluoroscopic image without contrast agent injection from the fluoroscopic image with contrast agent injection to obtain blood flow information in the area.

[0003] In view of the relative position changes of blood vessels and structural tissues caused by physiological movements of patients during the angiography process, registration and other methods are currently often used to adjust the image displacement, reduce the artifacts caused by movement, and then subtract the angiography film and the registered image. The current registration method is: use the edge points of the image to confirm the feature points, and then use the temporal movement law of the feature points to confirm the control nodes that are ultimately used for radiation transformation. Finally, this method uses the acquired nodes for registration and subtraction to obtain a subtraction map. However, this method that uses spatiotemporal information for feature point confirmation and registration is difficult to achieve complete registration for parts with larger displacements. Summary of the invention

[0004] In view of this, the present application provides a subtraction image generation method, apparatus, storage medium and computer equipment, which help to improve the accuracy of subtraction image generation.

[0005] According to one aspect of the present application, a subtraction image generation method is provided, comprising:

[0006] Obtaining a target non-contrast frame image and a target contrast frame image corresponding to the target object, and calculating an image difference value;

[0007] If the image difference value is less than a preset difference threshold, the target non-contrast frame is registered to obtain a target registered image, and the target contrast frame image and the target registered image are subtracted to obtain a first target subtraction image of the target object;

[0008] If the image difference value is greater than or equal to a preset difference threshold, the target angiography frame image is input into a trained subtraction model to obtain a second target subtraction image of the target object.

[0009] Optionally, the image difference value includes a feature difference value between a first image feature corresponding to a target non-contrast frame image and a second image feature corresponding to a target contrast frame image, and the calculating of the image difference value includes:

[0010] Determine a first image feature corresponding to the target non-contrast frame image and a second image feature corresponding to the target contrast frame image, wherein the first image feature includes a first grayscale feature of the target non-contrast frame image, and the second image feature includes a second grayscale feature of the target contrast frame image;

[0011] Calculating a grayscale feature difference value between the target non-contrast frame image and the target contrast frame image according to the first grayscale feature and the second grayscale feature;

[0012] Wherein, the feature difference value includes the gray feature difference value, the preset difference threshold includes a preset gray difference threshold, the preset gray difference threshold has a value range of [0.01, 0.2], and the unit is percentage; or,

[0013] Determine a first image feature corresponding to the target non-contrast frame image and a second image feature corresponding to the target contrast frame image, wherein the first image feature includes a first structural edge feature of the target non-contrast frame image, and the second image feature includes a second structural edge feature of the target contrast frame image;

[0014] Calculating a structural feature difference value between the target non-contrast frame image and the target contrast frame image according to the first structural edge feature and the second structural edge feature;

[0015] Among them, the feature difference value includes the structural feature difference value, the preset difference threshold includes a preset structural difference threshold, the preset structural difference threshold is a grayscale threshold, and the value range is [0.01, 0.4], or the preset structural difference threshold is a position threshold, and the value range is [20, 200] pixels.

[0016] Optionally, the image difference value includes a deformation amount between a plurality of target angiography frame images; and the step of calculating the image difference value includes:

[0017] The optical flow method is used to calculate the speed and displacement of adjacent target contrast frame images to obtain the deformation amount between adjacent target contrast frame images in a plurality of target contrast frame images;

[0018] The preset difference threshold is a preset deformation threshold, and the value range of the preset deformation threshold is [5, 100] pixels.

[0019] Optionally, registering the target non-contrast frame to obtain a target registered image specifically includes:

[0020] Inputting the target non-contrast frame image and the target contrast frame image into a trained registration model to obtain target registration data, wherein the target registration data includes a matrix containing displacement information of each pixel point of the target non-contrast frame image deformed toward the target contrast frame image;

[0021] The target non-contrast frame image is registered according to the target registration data to obtain the target registered image.

[0022] Optionally, the registration model is trained based on a first sample non-contrast frame image and a first sample contrast frame image corresponding to the first sample object.

[0023] Optionally, the training process of the registration model includes:

[0024] Acquire at least one first sample non-contrast frame image and at least one first sample contrast frame image corresponding to the first sample object;

[0025] Pairing any first sample non-contrast frame image with any first sample contrast frame image to obtain a plurality of first sample image pairs;

[0026] Inputting any first sample image pair into the registration model to be trained to obtain deformation data, registering the first sample non-contrast frame image in any first sample image pair according to the deformation data, and obtaining a first sample registered image corresponding to the any first sample image pair;

[0027] According to the feature difference value between the first sample registration image and the first sample angiography frame image corresponding to any first sample image pair, the registration model is iterated using a registration loss function until the training of the registration model is completed when a preset registration condition is met.

[0028] Optionally, the subtraction model is trained based on a second sample angiography frame image and a second sample subtraction image corresponding to the second sample object.

[0029] According to another aspect of the present application, a subtraction image generating device is provided, comprising:

[0030] A target image acquisition module is used to acquire a target non-contrast frame image and a target contrast frame image corresponding to the target object, and calculate an image difference value;

[0031] A first subtraction module is used for registering the target non-contrast frame to obtain a target registered image if the image difference value is less than a preset difference threshold, and subtracting the target contrast frame image from the target registered image to obtain a first target subtraction image of the target object;

[0032] The second subtraction module is used to input the target angiography frame image into a trained subtraction model to obtain a second target subtraction image of the target object if the image difference value is greater than or equal to a preset difference threshold.

[0033] Optionally, the target image acquisition module is specifically used to: determine a first image feature corresponding to the target non-contrast frame image and a second image feature corresponding to the target contrast frame image, wherein the first image feature includes a first grayscale feature of the target non-contrast frame image, and the second image feature includes a second grayscale feature of the target contrast frame image; calculate a grayscale feature difference value between the target non-contrast frame image and the target contrast frame image based on the first grayscale feature and the second grayscale feature; wherein the feature difference value includes the grayscale feature difference value, the preset difference threshold includes a preset grayscale difference threshold, and the preset grayscale difference threshold has a value range of [0.01, 0.2], and the unit is percentage; or,

[0034] Used to determine a first image feature corresponding to the target non-contrast frame image and a second image feature corresponding to the target contrast frame image, wherein the first image feature includes a first structural edge feature of the target non-contrast frame image, and the second image feature includes a second structural edge feature of the target contrast frame image; based on the first structural edge feature and the second structural edge feature, a structural feature difference value between the target non-contrast frame image and the target contrast frame image is calculated; wherein the feature difference value includes the structural feature difference value, the preset difference threshold includes a preset structural difference threshold, the preset structural difference threshold is a grayscale threshold, and the value range is [0.01, 0.4], or the preset structural difference threshold is a position threshold, and the value range is [20, 200] pixels.

[0035] Optionally, the first subtraction module also: inputs the target non-contrast frame image and the target contrast frame image into a trained registration model to obtain target registration data, wherein the target registration data includes a matrix containing displacement information of each pixel point of the target non-contrast frame image deformed toward the target contrast frame image; and registers the target non-contrast frame image according to the target registration data to obtain the target registered image.

[0036] Optionally, the registration model is trained based on a first sample non-contrast frame image and a first sample contrast frame image corresponding to the first sample object.

[0037] Optionally, the device further comprises:

[0038] A training module is used to obtain at least one first sample non-contrast frame image and at least one first sample contrast frame image corresponding to the first sample object; pair any first sample non-contrast frame image with any first sample contrast frame image to obtain multiple first sample image pairs; input any first sample image pair into a registration model to be trained to obtain deformation data, and register the first sample non-contrast frame image in any first sample image pair according to the deformation data to obtain a first sample registered image corresponding to any first sample image pair; and iterate the registration model using a registration loss function based on a feature difference value between the first sample registered image and the first sample contrast frame image corresponding to any first sample image pair, until the training of the registration model is completed when a preset registration condition is met.

[0039] Optionally, the subtraction model is trained based on a second sample angiography frame image and a second sample subtraction image corresponding to the second sample object.

[0040] According to another aspect of the present application, a storage medium is provided, on which a computer program is stored, and the program implements the above method when executed by a processor.

[0041] According to another aspect of the present application, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor implements the above method when executing the program.

[0042] By means of the above technical scheme, the present application provides a subtraction image generation method, device, storage medium and computer equipment, and the subtraction image generation method includes: after obtaining the target non-contrast frame image and the target contrast frame image of the target object, the image difference value is calculated, so as to identify the degree of motion artifact of the image according to the image difference value. When the degree of motion artifact is small, the target registration data is obtained through the pre-trained registration model, the target non-contrast frame image is registered, and then the target contrast frame image and the target registration image obtained by the registration are subtracted to generate a subtraction image; when the degree of motion artifact is large, the subtraction image is directly obtained using the pre-trained subtraction model. Compared with the current registration method based on spatiotemporal information, the embodiment of the present application can quickly and real-time generate a subtraction image as long as the target non-contrast frame image and the target contrast frame image are obtained, and for the case where the motion artifact is large, the subtraction image is directly generated using the pre-trained subtraction model, which solves the problem of poor registration effect in this case.

[0043] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0045] Figure 1 A schematic diagram of a process of generating a subtraction image provided by an embodiment of the present application is shown;

[0046] Figure 2 A schematic diagram of a process for generating a registration model provided in an embodiment of the present application is shown;

[0047] Figure 3 A schematic diagram of a workflow of a registration model provided in an embodiment of the present application is shown;

[0048] Figure 4 A schematic flow chart of a subtraction model generation method provided in an embodiment of the present application is shown;

[0049] Figure 5 A schematic diagram of a workflow of a subtraction model provided in an embodiment of the present application is shown;

[0050] Figure 6 A schematic diagram of the workflow of another subtraction model provided in an embodiment of the present application is shown;

[0051] Figure 7 A schematic diagram of a workflow of another subtraction model provided in an embodiment of the present application is shown;

[0052] Figure 8 A schematic structural diagram of a subtraction image generating device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0053] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

[0054] In this embodiment, a method for generating a subtraction image is provided, such as Figure 1 As shown, the method includes:

[0055] Step 101, obtaining a target non-contrast frame image and a target contrast frame image corresponding to a target object, and calculating an image difference value, wherein the image difference value includes a feature difference value between the target non-contrast frame image and the target contrast frame image;

[0056] The embodiments of the present application can be applied to a digital subtraction angiography (DSA) device to directly generate a subtraction image using images captured by the device. It can also be applied to a computer device independent of the DSA device to process images captured by the angiography device to generate a subtraction image.

[0057] First, during contrast imaging, contrast agents are usually injected after a delay of several frames, and the image without contrast agents is used as a mask, i.e., the target non-contrast frame image, and the frame image containing contrast agents is used as an imaging film, i.e., the target contrast frame image. After obtaining the target non-contrast frame image and the target contrast frame image of the target object, the image difference value can be obtained in the following way to judge the degree of motion artifacts, so as to select the corresponding subtraction method according to the degree of artifacts.

[0058] Optionally, the image difference value includes a feature difference value between a first image feature corresponding to the target non-contrast frame image and a second image feature corresponding to the target contrast frame image.

[0059] In one example, the “calculating image difference values” in step 101 may specifically include:

[0060] Step 101-1, determining a first image feature corresponding to the target non-contrast frame image and a second image feature corresponding to the target contrast frame image, wherein the first image feature includes a first grayscale feature of the target non-contrast frame image, and the second image feature includes a second grayscale feature of the target contrast frame image;

[0061] Step 101-2, calculating a grayscale feature difference value between the target non-contrast frame image and the target contrast frame image according to the first grayscale feature and the second grayscale feature;

[0062] Among them, the feature difference value includes the grayscale feature difference value, the preset difference threshold includes a preset grayscale difference threshold, the preset grayscale difference threshold has a value range of [0.01, 0.2], and the unit is percentage, that is, the grayscale difference threshold is the ratio of the difference between the grayscale mean of the target non-contrast frame image and the grayscale mean of the target contrast frame image to the grayscale mean of the target non-contrast frame image.

[0063] In another example, the “calculating image difference values” in step 101 may specifically include:

[0064] Step 101-3, determining a first image feature corresponding to the target non-contrast frame image and a second image feature corresponding to the target contrast frame image, wherein the first image feature includes a first structural edge feature of the target non-contrast frame image, and the second image feature includes a second structural edge feature of the target contrast frame image;

[0065] Step 101-4, calculating a structural feature difference value between the target non-contrast frame image and the target contrast frame image according to the first structural edge feature and the second structural edge feature;

[0066] Among them, the feature difference value includes the structural feature difference value, the preset difference threshold includes a preset structural difference threshold, the preset structural difference threshold is a grayscale threshold, and the value range is [0.01, 0.4], the unit is percentage, that is, the structural difference threshold is the ratio of the difference between the grayscale mean of the edge feature of the target non-contrast frame image and the grayscale mean of the edge feature of the target contrast frame image to the grayscale mean of the edge feature of the target non-contrast frame image, or the preset structural difference threshold is a position threshold, and the value range is [20, 200] pixels, that is, the structural difference threshold is the distance between the pixel position of the edge feature of the target non-contrast frame image and the pixel position of the edge feature of the target contrast frame image, the deformation field of the contrast image and the non-contrast image can be calculated by optical flow method or the like, and the deformation degree of the image can be calculated by using the displacement of the deformation field.

[0067] In this embodiment, the image features of the non-anatomical film and the angiographic film are calculated respectively, including but not limited to the grayscale features of the image, the structural edge features, etc. The feature difference between the mask and the angiographic film is calculated to determine the degree of motion artifacts, wherein the feature difference is reflected by the grayscale feature difference value and the structural feature difference value, the grayscale feature difference value can also be the grayscale mean square error between each pixel point of the mask and the angiographic film, and the structural feature difference value can be the position difference of the structural edge feature, etc.

[0068] Optionally, the image difference value includes a deformation amount between a plurality of target angiography frame images, and the “calculating the image difference value” in step 101 may specifically include:

[0069] The optical flow method is used to calculate the speed and displacement of adjacent target contrast frame images to obtain the deformation amount between adjacent target contrast frame images in a plurality of target contrast frame images;

[0070] The preset difference threshold is a preset deformation threshold, and the value range of the preset deformation threshold is [5, 100] pixels.

[0071] Step 102, if the image difference value is less than a preset difference threshold, register the target non-contrast frame to obtain a target registered image, and subtract the target contrast frame image from the target registered image to obtain a first target subtraction image of the target object;

[0072] In the above embodiment, if the image difference value is less than the preset difference threshold, it means that the degree of motion artifact is small. In this case, the mask can be registered first to make the tissue structure edge corresponding to the mask close to the angiogram, and then subtraction can be performed.

[0073] Optionally, in step 102, "registering the target non-contrast frame to obtain a target registered image" may specifically include: inputting the target non-contrast frame image and the target contrast frame image into a trained registration model to obtain target registration data, wherein the target registration data includes a matrix containing displacement information of each pixel point, in which the target non-contrast frame image is deformed toward the target contrast frame image; and registering the target non-contrast frame image according to the target registration data to obtain the target registered image.

[0074] In the above embodiment, the target non-contrast frame image and the target contrast frame image can be input into a pre-trained registration model, and the model output is used as the target registration data, which is a matrix containing the displacement information of each pixel point for deforming the target non-contrast frame image to the target contrast frame image. Thus, the target non-contrast frame image is registered according to the target registration data to obtain a target registered image and improve human body motion artifacts. Further, the target contrast frame image and the registered non-contrast frame image (i.e., the target registered image) are subtracted to obtain the final subtraction image, i.e., the first target subtraction image.

[0075] Among them, regarding the registration model, the first sample non-contrast frame image and the first sample contrast frame image corresponding to the first sample object can be used for training to ensure that the trained registration model can output the registration data of the mask according to the input mask and angiogram, and after the mask is registered according to the registration data, the feature difference value between the obtained registration image and the angiogram can be guaranteed to be within a certain range.

[0076] Step 103: If the image difference value is greater than or equal to a preset difference threshold, the target angiography frame image is input into a trained subtraction model to obtain a second target subtraction image of the target object.

[0077] In the above embodiment, if the feature difference value is greater than or equal to the preset difference threshold, it means that the degree of motion artifact is large. In this case, due to the large degree of displacement of the tissue structure, the way of registering the mask may be difficult to ensure accuracy. For example, the registration may cause the tissue structure image to be unnatural. In this scenario, by pre-training a subtraction image generation model (referred to as the subtraction model), the target angiography frame image can be input into the trained subtraction model to directly generate a subtraction image, thereby avoiding the problem of poor subtraction image effect due to the difficulty in ensuring the registration effect due to the large motion artifact, and improving the accuracy of the subtraction image.

[0078] Among them, regarding the subtraction model, the second sample angiography frame image and the second sample subtraction map corresponding to the second sample object can be used for training, so that the trained subtraction model can directly output the subtraction map according to the input angiography film. Specifically, a second sample angiography frame image and a second sample non-angiography frame image with a very small feature difference value can be selected, and the second sample subtraction map is obtained by subtracting the two to ensure that the subtraction map is accurate and reliable. Of course, the feature difference value between the second sample non-angiography frame image and the second sample angiography frame image may not be limited, as long as the second sample subtraction map can be guaranteed to be reliable.

[0079] By applying the technical solution of this embodiment, after obtaining the target non-contrast frame image and the target contrast frame image of the target object, the feature difference value between the two is calculated, or the deformation variable of the target contrast frame image is calculated, that is, the image difference value is calculated, so as to identify the degree of motion artifact of the image according to the image difference value. When the degree of motion artifact is small, the target registration data is obtained through the pre-trained registration model, the target non-contrast frame image is registered, and then the target contrast frame image and the registered target registration image are subtracted to generate a subtraction map; when the degree of motion artifact is large, the subtraction map is directly obtained using the pre-trained subtraction model. Compared with the current registration method based on spatiotemporal information, the embodiment of the present application can quickly and real-time generate a subtraction map as long as the target non-contrast frame image and the target contrast frame image are obtained, and for the case where the motion artifact is large, the subtraction map is directly generated using the pre-trained subtraction model, which solves the problem of poor registration effect in this case.

[0080] In this embodiment, a registration model generation method is provided, such as Figure 2 As shown, the method includes:

[0081] Step 201, obtaining at least one first sample non-contrast frame image and at least one first sample contrast frame image corresponding to a first sample object;

[0082] Step 202, pairing any first sample non-contrast frame image with any first sample contrast frame image to obtain a plurality of candidate image pairs, and selecting the candidate image pairs whose feature difference values ​​are greater than a preset first sample threshold as first sample image pairs;

[0083] Step 203, inputting any first sample image pair into the registration model to be trained to obtain deformation data, registering the first sample non-contrast frame image in any first sample image pair according to the deformation data, and obtaining a first sample registered image corresponding to the any first sample image pair;

[0084] Step 204, based on the feature difference value between the first sample registered image and the first sample angiography frame image corresponding to any first sample image pair, the registration model is iterated using a registration loss function until the registration model is trained when a preset registration condition is met.

[0085] In an embodiment of the present application, a first sample non-contrast frame image and a first sample contrast frame image of a first sample object collected by a DSA device are obtained, and the collected images are paired to form an image pair. To ensure the model training effect and enable the trained registration model to effectively register image motion artifacts, the paired image pairs can be screened. For any image pair, the feature difference value between the two images contained in the image pair should be greater than the preset first sample threshold, thereby ensuring effective training of the model. Among them, the first sample image pair can be composed of a first sample non-contrast frame image and a first sample contrast frame image, or it can be composed of two first sample contrast frame images, as long as the feature difference value between the two images in the first sample image pair finally used for training is greater than the above threshold. In order to improve the pairing efficiency, one image can be directly selected from multiple first sample non-contrast frame images and one image can be directly selected from multiple first sample contrast frame images to form an image pair.

[0086] like Figure 3As shown, a registration neural network model to be trained is constructed. For example, a neural network such as a densely connected convolutional network can be selected as a registration model. The first sample image pair is input into the registration model. Using the network, the image pair can be generated into deformation data after network operation. The deformation data is a matrix containing displacement information of each pixel point for deforming the first sample non-contrast frame image to the first sample contrast frame image. The first sample non-contrast frame image is registered by the obtained deformation data, so that the first sample non-contrast frame image is deformed into the first sample registration map. The feature difference value between the first sample registration map and the first sample contrast frame image is further calculated, and the registration loss function of the registration model is determined based on the feature difference value. The registration model is iterated using the registration loss function until the registration loss function of the registration model is stabilized within a certain range, and the training is completed.

[0087] In the embodiment of the present application, optionally, step 204 may specifically include:

[0088] Step 204 - 1 : taking the feature difference value between the first sample registered image and the first sample angiography frame image corresponding to any first sample image pair as the registration loss function of the registration model.

[0089] In the above embodiment, an optional way to determine the loss function is to directly use the feature difference value between the first sample registration image and the first sample angiography frame image as the registration loss function.

[0090] In the embodiment of the present application, optionally, step 204 may specifically include:

[0091] Step 204-2, using the blood vessel extraction model, determining a first sample blood vessel segmentation image of a first sample angiography frame image in any first sample image pair, and down-weighting corresponding pixels of the first sample registration image based on the first sample blood vessel segmentation image;

[0092] Step 204-3, using the feature difference value between the down-weighted first sample registration image and the first sample angiography frame image corresponding to any first sample image pair as the down-weighted perceptual loss function of the registration model;

[0093] Step 204 - 4 , determining the registration loss function of the registration model according to the reduced-weighted perceptual loss function and the regularization term corresponding to the registration model.

[0094] In the above embodiment, in order to solve the problem of unnatural graphics caused by registration, and the problem that the blood vessels and other regions are difficult to register due to the different image information caused by the absence of blood vessels and other tissue structures in the mask but the presence of blood vessels and other tissue structures in the angiogram (this problem can also be understood as the image information is lost because the tissue structure in the angiogram is blocked by the structure after the angiography, and this part cannot be matched through the image information), the registration loss function of the model can be determined by the regularization term + tissue deweighted perception loss. The embodiment of the present application takes the down-weighting of blood vessels as an example, wherein the down-weighting loss function for blood vessels can be obtained through a pre-trained blood vessel extraction network, and the specific steps are: use a pre-trained blood vessel extraction model, such as a blood vessel segmentation network, to perform blood vessel segmentation on the input first sample angiography frame image, and generate a segmentation result, i.e., a first sample blood vessel segmentation image; then, use the first sample blood vessel segmentation image to down-weight the first sample registration image, and specifically record the pixels where blood vessels exist in the first sample blood vessel segmentation image, and use these pixel information to down-weight the pixels corresponding to the first sample registration image (i.e., multiply by a coefficient of <1), so as to reduce the impact caused by blood vessels; finally, calculate the feature difference value between the down-weighted first sample registration image and the first sample angiography frame image, and use this value as the down-weighted perceptual loss function of the model. Furthermore, in order to avoid the over-fitting phenomenon of the model, the down-weighted perceptual loss function is added to the regularization term corresponding to the model, and the sum of the two is used as the registration loss function of the model. Among them, the expression of the regularization term can be: Among them, L all is the overall loss function, L(x) represents the loss function of calculating the difference between the target and the target, such as perceptual loss, L2 loss, etc., λ is the weight of the L2 regularization term, which represents the proportion of the regularization term in the loss function; x-x' is the gradient value of the image at pixel i and the surrounding pixels. is the L2 regularization penalty term, which calculates the sum of the squares of the gradients of all pixels in the image.

[0095] In the embodiment of the present application, the registration model trained in the above manner can realize rapid registration of the mask after the mask and the angiogram are collected. In practical applications, the trained registration model is read, the mask and the angiogram to be registered are input into the model, the registration data output by the model is applied to the mask for registration, a registration image is generated, and the angiogram and the registration image are subtracted to generate a subtraction image.

[0096] In this embodiment, a subtraction model generation method is provided, such as Figure 4 As shown, the method includes:

[0097] Step 301, obtaining at least one second sample non-contrast frame image and at least one second sample contrast frame image corresponding to a second sample object, wherein a feature difference value between the second sample non-contrast frame image and the second sample contrast frame image is less than a preset second sample threshold;

[0098] Step 302, subtracting any second sample contrast frame image from any second sample non-contrast frame image to obtain a second sample subtraction image, and pairing the any second sample contrast frame image with the second sample subtraction image to form a second sample image pair;

[0099] Step 303: Use the second sample image pair to train the subtraction model to be trained to obtain a trained subtraction model.

[0100] In the above embodiment, in the images collected by the DSA device, a sequence with basically no motion artifacts is selected as a training data set. Specifically, the second sample non-contrast frame image and the second sample contrast frame image corresponding to the second sample object can be obtained, and the sample data is screened according to the feature difference value to ensure that the feature difference value between any second sample non-contrast frame image and any second sample contrast frame image is less than the preset second sample threshold value, and the preset second sample threshold value is determined with reference to the preset grayscale difference threshold value and the preset deformation value threshold value. In a specific application scenario, since there is basically no motion artifact, step 301 can only select one second sample non-contrast frame image to reduce the subsequent calculation amount. Then, the second sample contrast frame image and the second sample non-contrast frame image are subtracted to obtain a second sample subtraction image, and the second sample subtraction image and the corresponding second sample contrast frame image are paired to form a second sample image pair as a model training sample. Finally, a subtraction model is constructed, and the second sample image pair is used for training. Specifically, the second sample contrast frame image can be used as the model input, and the second sample subtraction image can be used as the model output to achieve the training of the model. Among them, the subtraction model can select a spatial pyramid network with densely connected convolutions.

[0101] In the embodiment of the present application, optionally, step 303 may specifically include:

[0102] Step 303-1, input the second sample angiography frame image in the second sample image pair into the subtraction model to be trained to obtain the second sample subtraction prediction image, and determine the subtraction loss function of the subtraction model according to the feature difference value between the second sample subtraction image and the second sample subtraction prediction image; use the subtraction loss function to iterate the subtraction model until the training of the subtraction model is completed when the preset subtraction condition is met.

[0103] In the above embodiment, if Figure 5As shown, during the training process, for any second sample image pair, the second sample angiography frame image is input into the subtraction model to be trained to obtain the second sample subtraction prediction image, and then the feature difference value between the second sample subtraction prediction image and the second sample subtraction image in the second sample image pair is calculated, and the value is used as the subtraction loss function of the subtraction model. The obtained subtraction loss function is used to iterate the model network parameters until the subtraction loss function is stable within a certain range, and the training of the subtraction model is completed. It should be noted that the subtraction loss function can also select the vascular down-weighted perception loss, and the specific calculation method is similar to that of step 204-2 and step 204-3, which will not be repeated here.

[0104] In the embodiment of the present application, optionally, step 303 may specifically include:

[0105] Step 303-2, if the number of the second sample image pairs is less than the preset sample number, obtaining a plurality of third sample image pairs corresponding to the third sample object, wherein any third sample image pair includes a third sample angiography frame image and a third sample subtraction image of the third sample object, and the third sample object corresponds to the second part;

[0106] Step 303-3, training a preset neural network model according to the third sample image pair to obtain an auxiliary model;

[0107] Step 303 - 4 , establishing the subtraction model to be trained based on the auxiliary model and the preset neural network model.

[0108] In the above embodiment, in view of the problem that there are few motion artifact-free images of certain parts (such as the heart) in the DSA image sequence and insufficient training samples, the embodiment of the present application also provides a solution based on transfer learning that can be used in the design of the subtraction model. In a specific application scenario, samples without motion artifacts of some different parts, such as samples of the head and abdomen, can be used first, and the preset neural network model training can be completed in the manner of steps 301 to 303 above to obtain an auxiliary model, thereby constructing a subtraction model to be trained based on the auxiliary model and the initial preset neural network model.

[0109] Optionally, step 303-4 may specifically include: obtaining a preset migration layer in the auxiliary model as a migration model; inputting the second sample contrast frame image in the second sample image pair into the migration model to obtain migration data corresponding to the second sample contrast frame image, and constructing a second sample migration data pair including the migration data and the corresponding second sample subtraction image; obtaining a preset migrated layer in the initial preset neural network model as a migrated model, and training the migrated model using the second sample migration data pair; splicing the migration model and the migrated model to obtain the subtraction model.

[0110] Among them, a part of the front network layer is extracted from the trained neural network model, i.e., the auxiliary model, and the extracted network structure is used as the migration model, such as Figure 6 As shown, the network layer to be extracted can be agreed upon, that is, a preset migration layer is set, and then the second sample contrast frame image in the second sample image pair is input into the migration model to obtain the corresponding migration data, that is, the feature matrix of the second sample contrast frame image is generated using the middle layer of the pre-trained neural network model (that is, the preset migration layer). Thus, the migrated part corresponding to the subtraction model is trained using the migration data as a training sample. Specifically, the preset migrated layer in the preset neural network model can be obtained as the migrated model, so that the migrated model is trained using the migration data and the second sample subtraction image corresponding to the migration data, thereby reducing the number of layers of model training and the amount of network parameter data that needs to be determined. In this way, the network parameters of the migration model do not need to be changed, and only the parameters of the remaining network layers (that is, the preset migrated layers) need to be trained. Finally, based on the same network structure, the migration model and the migrated model can be spliced ​​together to obtain the final subtraction model.

[0111] In an embodiment of the present application, optionally, step 303-4 may specifically include: cascading the auxiliary model to the preset neural network model to obtain the subtraction model to be trained, and using the second sample image pair to train the subtraction model to be trained.

[0112] In the above embodiment, if Figure 7 As shown, after determining the fourth sample image pair, the fourth sample image pair is used to train the preset neural network model, and then the trained preset neural network model is cascaded to the initial preset neural network model to determine the subtraction model to be trained. Specifically, the preset neural network model can be divided into two parts according to the network layer, namely subtraction network 1 and subtraction network 2, which are cascaded to the middle position between the two, and then the second sample image pair is continued to be used for model training.

[0113] Further, as Figure 1 The specific implementation of the method, the embodiment of the present application provides a subtraction image generation device, such as Figure 8 As shown, the device comprises:

[0114] A target image acquisition module is used to acquire a target non-contrast frame image and a target contrast frame image corresponding to the target object, and calculate an image difference value;

[0115] A first subtraction module is used for registering the target non-contrast frame to obtain a target registered image if the image difference value is less than a preset difference threshold, and subtracting the target contrast frame image from the target registered image to obtain a first target subtraction image of the target object;

[0116] The second subtraction module is used to input the target angiography frame image into a trained subtraction model to obtain a second target subtraction image of the target object if the image difference value is greater than or equal to a preset difference threshold.

[0117] Optionally, the target image acquisition module can be specifically used to: determine a first image feature corresponding to the target non-contrast frame image and a second image feature corresponding to the target contrast frame image, wherein the first image feature includes a first grayscale feature of the target non-contrast frame image, and the second image feature includes a second grayscale feature of the target contrast frame image; based on the first grayscale feature and the second grayscale feature, calculate a grayscale feature difference value between the target non-contrast frame image and the target contrast frame image; wherein the feature difference value includes the grayscale feature difference value, and the preset difference threshold includes a preset grayscale difference threshold, and the preset grayscale difference threshold has a value range of [0.01,0.2], and the unit is percentage.

[0118] Optionally, the target image acquisition module can be specifically used to: determine a first image feature corresponding to the target non-contrast frame image and a second image feature corresponding to the target contrast frame image, wherein the first image feature includes a first structural edge feature of the target non-contrast frame image, and the second image feature includes a second structural edge feature of the target contrast frame image; based on the first structural edge feature and the second structural edge feature, calculate a structural feature difference value between the target non-contrast frame image and the target contrast frame image; wherein the feature difference value includes the structural feature difference value, and the preset difference threshold includes a preset structural difference threshold, and the preset structural difference threshold is a grayscale threshold with a value range of [0.01, 0.4], or the preset structural difference threshold is a position threshold with a value range of [20, 200] pixels.

[0119] Optionally, the target image acquisition module can be specifically used to: use the optical flow method to calculate the speed and displacement of adjacent target imaging frame images, and obtain the deformation amount between adjacent target imaging frame images in multiple target imaging frame images; wherein the preset difference threshold is a preset deformation amount threshold, and the value range of the preset deformation amount threshold is [5, 100] pixels.

[0120] Optionally, the first subtraction module can be used to: input the target non-contrast frame image and the target contrast frame image into a trained registration model to obtain target registration data, wherein the target registration data includes a matrix containing the displacement information of each pixel point of the target non-contrast frame image deformed toward the target contrast frame image; and register the target non-contrast frame image according to the target registration data to obtain the target registered image.

[0121] Optionally, the registration model is trained based on a first sample non-contrast frame image and a first sample contrast frame image corresponding to the first sample object.

[0122] Optionally, the device further comprises:

[0123] A training module is used to obtain at least one first sample non-contrast frame image and at least one first sample contrast frame image corresponding to the first sample object; pair any first sample non-contrast frame image with any first sample contrast frame image to obtain multiple first sample image pairs; input any first sample image pair into a registration model to be trained to obtain deformation data, and register the first sample non-contrast frame image in any first sample image pair according to the deformation data to obtain a first sample registered image corresponding to any first sample image pair; and iterate the registration model using a registration loss function based on a feature difference value between the first sample registered image and the first sample contrast frame image corresponding to any first sample image pair, until the training of the registration model is completed when a preset registration condition is met.

[0124] Optionally, the subtraction model is trained based on a second sample angiography frame image and a second sample subtraction image corresponding to the second sample object.

[0125] It should be noted that for other corresponding descriptions of the functional units involved in the subtraction image generation device provided in the embodiment of the present application, reference can be made to Figures 1 to 7 The corresponding description in the method will not be repeated here.

[0126] Based on the above Figures 1 to 7 The method shown in the embodiment of the present application is accordingly provided with a storage medium on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned Figures 1 to 7 The subtraction image generation method shown.

[0127] Based on this understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each implementation scenario of the present application.

[0128] Based on the above Figures 1 to 7 The method shown in the figure, and the virtual device embodiment, in order to achieve the above purpose, the embodiment of the present application also provides a computer device, which can be a personal computer, a server, a network device, etc., and the computer device includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to achieve the above Figures 1 to 7 The subtraction image generation method shown.

[0129] Optionally, the computer device may further include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a WI-FI module, etc. The user interface may include a display, an input unit such as a keyboard, etc., and the optional user interface may also include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (such as a Bluetooth interface, a WI-FI interface), etc.

[0130] Those skilled in the art will appreciate that the computer device structure provided in this embodiment does not limit the computer device, and may include more or fewer components, or a combination of certain components, or different component arrangements.

[0131] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages and saves the hardware and software resources of the computer device, and supports the operation of information processing programs and other software and / or programs. The network communication module is used to realize communication between the components inside the storage medium, and communication with other hardware and software in the physical device.

[0132] Through the description of the above implementation methods, those skilled in the art can clearly understand that the present application can be implemented by means of software plus the necessary general hardware platform, or by hardware implementation to obtain the target non-contrast frame image and the target contrast frame image of the target object, and then calculate the image difference value, so as to identify the degree of motion artifacts of the image based on the image difference value. When the degree of motion artifacts is small, the target registration data is obtained through a pre-trained registration model, the target non-contrast frame image is registered, and then the target contrast frame image and the registered target registration image are subtracted to generate a subtraction map; when the degree of motion artifacts is large, the subtraction map is directly obtained using a pre-trained subtraction model. Compared with the current registration method based on spatiotemporal information, the embodiment of the present application can quickly and real-time generate a subtraction map as long as the target non-contrast frame image and the target contrast frame image are obtained, and for the case where the motion artifacts are large, the subtraction map is directly generated using a pre-trained subtraction model, which solves the problem of poor registration effect in this case.

[0133] Those skilled in the art will appreciate that the accompanying drawings are only schematic diagrams of a preferred implementation scenario, and the modules or processes in the accompanying drawings are not necessarily necessary for implementing the present application. Those skilled in the art will appreciate that the modules in the devices in the implementation scenario can be distributed in the devices of the implementation scenario according to the description of the implementation scenario, or can be changed accordingly and located in one or more devices different from the present implementation scenario. The modules of the above-mentioned implementation scenario can be combined into one module, or can be further split into multiple submodules.

[0134] The above serial numbers of this application are only for description and do not represent the advantages and disadvantages of the implementation scenarios. The above disclosure is only a few specific implementation scenarios of this application, but this application is not limited to them, and any changes that can be thought of by technicians in this field should fall within the scope of protection of this application.

Claims

1. A method for generating a subtraction image, characterized in that: include: Obtaining a target non-contrast frame image and a target contrast frame image corresponding to the target object, and calculating an image difference value; If the image difference value is less than a preset difference threshold, the target non-contrast frame is registered to obtain a target registered image, and the target contrast frame image and the target registered image are subtracted to obtain a first target subtraction image of the target object; If the image difference value is greater than or equal to a preset difference threshold, the target angiography frame image is input into a trained subtraction model to obtain a second target subtraction image of the target object.

2. The method according to claim 1, characterized in that The image difference value includes a feature difference value between a first image feature corresponding to a target non-contrast frame image and a second image feature corresponding to a target contrast frame image, and the image difference value is calculated, including: Determine a first image feature corresponding to the target non-contrast frame image and a second image feature corresponding to the target contrast frame image, wherein the first image feature includes a first grayscale feature of the target non-contrast frame image, and the second image feature includes a second grayscale feature of the target contrast frame image; Calculating a grayscale feature difference value between the target non-contrast frame image and the target contrast frame image according to the first grayscale feature and the second grayscale feature; Wherein, the feature difference value includes the gray feature difference value, the preset difference threshold includes a preset gray difference threshold, the preset gray difference threshold has a value range of [0.01, 0.2], and the unit is percentage; or, Determine a first image feature corresponding to the target non-contrast frame image and a second image feature corresponding to the target contrast frame image, wherein the first image feature includes a first structural edge feature of the target non-contrast frame image, and the second image feature includes a second structural edge feature of the target contrast frame image; Calculating a structural feature difference value between the target non-contrast frame image and the target contrast frame image according to the first structural edge feature and the second structural edge feature; Among them, the feature difference value includes the structural feature difference value, the preset difference threshold includes a preset structural difference threshold, the preset structural difference threshold is a grayscale threshold, and the value range is [0.01, 0.4], or the preset structural difference threshold is a position threshold, and the value range is [20, 200] pixels.

3. The method according to claim 1, characterized in that The image difference value includes the deformation amount between a plurality of target angiography frame images; The calculating of the image difference value comprises: The optical flow method is used to calculate the speed and displacement of adjacent target contrast frame images to obtain the deformation amount between adjacent target contrast frame images in a plurality of target contrast frame images; The preset difference threshold is a preset deformation threshold, and the value range of the preset deformation threshold is [5, 100] pixels.

4. The method according to claim 1, characterized in that: The registering the target non-contrast frame to obtain a target registered image specifically includes: Inputting the target non-contrast frame image and the target contrast frame image into a trained registration model to obtain target registration data, wherein the target registration data includes a matrix containing displacement information of each pixel point of the target non-contrast frame image deformed toward the target contrast frame image; The target non-contrast frame image is registered according to the target registration data to obtain the target registered image.

5. The method according to claim 4, characterized in that The registration model is trained based on a first sample non-contrast frame image and a first sample contrast frame image corresponding to a first sample object.

6. The method according to claim 5, characterized in that The training process of the registration model includes: Acquire at least one first sample non-contrast frame image and at least one first sample contrast frame image corresponding to the first sample object; Pairing any first sample non-contrast frame image with any first sample contrast frame image to obtain a plurality of first sample image pairs; Inputting any first sample image pair into the registration model to be trained to obtain deformation data, registering the first sample non-contrast frame image in any first sample image pair according to the deformation data, and obtaining a first sample registered image corresponding to the any first sample image pair; According to the feature difference value between the first sample registration image and the first sample angiography frame image corresponding to any first sample image pair, the registration model is iterated using a registration loss function until the training of the registration model is completed when a preset registration condition is met.

7. The method according to claim 1, characterized in that The subtraction model is trained based on the second sample angiography frame image and the second sample subtraction image corresponding to the second sample object.

8. A subtraction image generating device, characterized in that: include: A target image acquisition module is used to acquire a target non-contrast frame image and a target contrast frame image corresponding to the target object, and calculate an image difference value; A first subtraction module is used for registering the target non-contrast frame to obtain a target registered image if the image difference value is less than a preset difference threshold, and subtracting the target contrast frame image from the target registered image to obtain a first target subtraction image of the target object; The second subtraction module is used to input the target angiography frame image into a trained subtraction model to obtain a second target subtraction image of the target object if the image difference value is greater than or equal to a preset difference threshold.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

Images