Subtraction image generation method and device, storage medium, and computer equipment

Abstract

The present application discloses a subtraction image generation method and device, storage medium, and computer equipment, the method comprising: obtaining a target contrast frame image; inputting the target contrast frame image into a trained subtraction model to obtain a target subtraction image corresponding to the target contrast frame image, wherein the subtraction model is trained by a first sample contrast frame image and its corresponding first sample subtraction reference image. The present application helps to realize the real-time generation of subtraction images.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a subtraction image generation method and device, a storage medium, and a computer device. Background Art

[0002] Digital subtraction angiography is a new X-ray imaging system and one of the most commonly used devices for clinical precision treatment and guided interventional surgery. During angiography, the flow state of the contrast agent is observed by subtracting the fluoroscopic image without contrast agent injection from the fluoroscopic image with contrast agent injection to obtain blood flow information in the area.

[0003] In view of the relative position changes of blood vessels and structural tissues caused by physiological movements of patients during the angiography process, registration and other methods are currently often used to adjust the image displacement, reduce the artifacts caused by movement, and then subtract the angiography film and the registered image. The current registration method is: use the edge points of the image to confirm the feature points, and then use the movement law of the feature points in time to confirm the control nodes that are ultimately used for radiation transformation. Finally, this method uses the acquired nodes for registration and subtraction to obtain a subtraction map. However, this method that uses spatiotemporal information for feature point confirmation and registration is difficult to achieve real-time performance. Summary of the invention

[0004] In view of this, the present application provides a subtraction image generation method and apparatus, a storage medium, and a computer device.

[0005] According to one aspect of the present application, a subtraction image generation method is provided, characterized in that it includes:

[0006] Acquire a target angiography frame image;

[0007] The target angiography frame image is input into a trained subtraction model to obtain a target subtraction image corresponding to the target angiography frame image, wherein the subtraction model is trained by a first sample angiography frame image and its corresponding first sample subtraction reference image.

[0008] Optionally, the acquiring of the target angiography frame image specifically includes:

[0009] Acquire a contrast-enhanced frame image to be subtracted and a corresponding non-contrast-enhanced frame image, and calculate a target feature difference value between the contrast-enhanced frame image to be subtracted and the non-contrast-enhanced frame image;

[0010] When the target feature difference value is greater than a preset difference threshold, the angiography frame image to be subtracted is used as the target angiography frame image.

[0011] Optionally, the calculating the target feature difference value between the contrast-enhanced frame image to be subtracted and the non-contrast-enhanced frame image specifically includes:

[0012] Determine a first image feature corresponding to the non-contrast frame image and a second image feature corresponding to the contrast frame image to be subtracted, wherein the first image feature includes a first grayscale feature of the non-contrast frame image, and the second image feature includes a second grayscale feature of the contrast frame image to be subtracted; calculate a grayscale feature difference value between the non-contrast frame image and the contrast frame image to be subtracted based on the first grayscale feature and the second grayscale feature; wherein the target feature difference value includes the grayscale feature difference value, the preset difference threshold includes a preset grayscale difference threshold, and the preset grayscale difference threshold has a value range of [0.01, 0.2], in percentage; or,

[0013] Determine a first image feature corresponding to the non-contrast frame image and a second image feature corresponding to the contrast frame image to be subtracted, wherein the first image feature includes a first structural edge feature of the target non-contrast frame image, and the second image feature includes a second structural edge feature of the contrast frame image to be subtracted; calculate a structural feature difference value between the non-contrast frame image and the contrast frame image to be subtracted based on the first structural edge feature and the second structural edge feature; wherein the target feature difference value includes the structural feature difference value, the preset difference threshold includes a preset structural difference threshold, the preset structural difference threshold is a grayscale threshold, and the value range is [0.01, 0.4], or the preset structural difference threshold is a position threshold, and the value range is [20, 200] pixels.

[0014] Optionally, the training step of the subtraction model includes:

[0015] Acquire a first sample image, wherein the first sample image includes at least one first sample non-contrast frame image and at least one first sample contrast frame image, and a feature difference value between the first sample non-contrast frame image and the first sample contrast frame image is less than a preset sample threshold;

[0016] Subtracting any first sample contrast frame image from any first sample non-contrast frame image to obtain a first sample subtraction reference image, and forming a first sample image pair with the any first sample contrast frame image and the corresponding first sample subtraction reference image;

[0017] The subtraction model to be trained is trained using the first sample image pair.

[0018] Optionally, the using the first sample image pair to train the subtraction model to be trained specifically includes:

[0019] The first sample angiography frame image in any first sample image pair is input into the subtraction model to be trained to obtain a first sample subtraction prediction image, and the subtraction model is iterated using a loss function based on the feature difference value between the first sample subtraction reference image and the first sample subtraction prediction image, until the loss function of the obtained subtraction model meets the preset subtraction condition, the training of the subtraction model is completed.

[0020] Optionally, before using the first sample image pair to train the subtraction model to be trained, the method further includes:

[0021] Acquire a plurality of second sample image pairs, wherein any second sample image pair comprises a second sample contrast frame image and a second sample subtraction reference image, and the second sample image pair corresponds to a second part, wherein the first part and the second part correspond to images of different positions of the sample;

[0022] Training a preset neural network model according to the second sample image pair to obtain an auxiliary model;

[0023] The subtraction model to be trained is established based on the auxiliary model and the preset neural network model, and the subtraction model to be trained is trained using the first sample image pair.

[0024] Optionally, establishing the subtraction model to be trained based on the auxiliary model and the preset neural network model, and training the subtraction model to be trained using the first sample image pair specifically includes:

[0025] Acquire a preset migration layer in the auxiliary model as a migration model;

[0026] Inputting the first sample contrast frame image in the first sample image pair into the migration model to obtain migration data corresponding to the first sample contrast frame image, and constructing a first sample migration data pair including the migration data and the corresponding first sample subtraction reference image;

[0027] Acquire a preset migrated layer in the initial preset neural network model as a migrated model, and train the migrated model using the second sample migration data;

[0028] The migration model and the trained migrated model are spliced ​​to obtain the subtraction model.

[0029] According to another aspect of the present application, a subtraction image generating device is provided, comprising:

[0030] A target image acquisition module is used to acquire a target angiography frame image;

[0031] The subtraction image generation module is used to input the target angiography frame image into a trained subtraction model to obtain a target subtraction image corresponding to the target angiography frame image, wherein the subtraction model is trained by a first sample angiography frame image and its corresponding first sample subtraction reference image.

[0032] Optionally, the target image acquisition module is specifically used to:

[0033] Acquire a contrast-enhanced frame image to be subtracted and a corresponding non-contrast-enhanced frame image, and calculate a target feature difference value between the contrast-enhanced frame image to be subtracted and the non-contrast-enhanced frame image;

[0034] When the target feature difference value is greater than a preset difference threshold, the angiography frame image to be subtracted is used as the target angiography frame image.

[0035] Optionally, the target image acquisition module is further used for:

[0036] Specifically used for: determining a first image feature corresponding to the non-contrast frame image and a second image feature corresponding to the target contrast frame image, wherein the first image feature includes a first grayscale feature of the non-contrast frame image, and the second image feature includes a second grayscale feature of the contrast frame image to be subtracted; calculating a grayscale feature difference value between the non-contrast frame image and the contrast frame image to be subtracted based on the first grayscale feature and the second grayscale feature; wherein the target feature difference value includes the grayscale feature difference value, the preset difference threshold includes a preset grayscale difference threshold, and the preset grayscale difference threshold has a value range of [0.01, 0.2], in percentage; or,

[0037] Used to determine a first image feature corresponding to the non-contrast frame image and a second image feature corresponding to the contrast frame image to be subtracted, wherein the first image feature includes a first structural edge feature of the non-contrast frame image, and the second image feature includes a second structural edge feature of the contrast frame image to be subtracted; based on the first structural edge feature and the second structural edge feature, a structural feature difference value between the non-contrast frame image and the contrast frame image to be subtracted is calculated; wherein the target feature difference value includes the structural feature difference value, the preset difference threshold includes a preset structural difference threshold, the preset structural difference threshold is a grayscale threshold, and the value range is [0.01, 0.4], or the preset structural difference threshold is a position threshold, and the value range is [20, 200] pixels.

[0038] Optionally, the device further comprises:

[0039] A model training module is used to obtain a first sample image, wherein the first sample image includes at least one first sample non-contrast frame image and at least one first sample contrast frame image, and the feature difference value between the first sample non-contrast frame image and the first sample contrast frame image is less than a preset sample threshold; subtract any first sample contrast frame image from any first sample non-contrast frame image to obtain a first sample subtraction reference image, and form a first sample image pair with any first sample contrast frame image and the corresponding first sample subtraction reference image; and use the first sample image pair to train the subtraction model to be trained.

[0040] Optionally, the model training module is specifically used to: input the first sample angiography frame image in any first sample image pair into the subtraction model to be trained to obtain a first sample subtraction prediction image, and iterate the subtraction model using a loss function based on the feature difference value between the first sample subtraction reference image and the first sample subtraction prediction image, until the loss function of the obtained subtraction model meets the preset subtraction condition, thereby completing the training of the subtraction model.

[0041] Optionally, the model training module is also used to:

[0042] Acquire a plurality of second sample image pairs, wherein any second sample image pair comprises a second sample contrast frame image and a second sample subtraction reference image, and the second sample image pair corresponds to a second part, wherein the first part and the second part correspond to images of different positions of the sample;

[0043] Training a preset neural network model according to the second sample image pair to obtain an auxiliary model;

[0044] The subtraction model to be trained is established based on the auxiliary model and the preset neural network model, and the subtraction model to be trained is trained using the first sample image pair.

[0045] Optionally, the model training module is also used to:

[0046] Acquire a preset migration layer in the auxiliary model as a migration model;

[0047] Inputting the first sample contrast frame image in the first sample image pair into the migration model to obtain migration data corresponding to the first sample contrast frame image, and constructing a first sample migration data pair including the migration data and the corresponding first sample subtraction reference image;

[0048] Acquire a preset migrated layer in the initial preset neural network model as a migrated model, and train the migrated model using the second sample migration data;

[0049] The migration model and the trained migrated model are spliced ​​to obtain the subtraction model.

[0050] According to another aspect of the present application, a storage medium is provided, on which a computer program is stored, and the program implements the above method when executed by a processor.

[0051] According to another aspect of the present application, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor implements the above method when executing the program.

[0052] By means of the above technical solution, the present application provides a subtraction image generation method and device, storage medium, and computer equipment, the method comprising: obtaining a target contrast frame image; inputting the target contrast frame image into a trained subtraction model to obtain a target subtraction image corresponding to the target contrast frame image, wherein the subtraction model is trained by a first sample contrast frame image and its corresponding first sample subtraction reference image. The present application can realize real-time registration of images, especially for the problem that the degree of displacement of tissue structure is large and registration easily leads to unnatural tissue structure images. The target contrast frame image is input into a trained subtraction model to directly generate a subtraction image, thereby avoiding the problem that the subtraction image effect is poor due to the difficulty in ensuring the registration effect due to excessive motion artifacts, and improving the accuracy of the subtraction image.

[0053] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0055] Figure 1 A schematic diagram of a process for generating a subtraction image provided by an embodiment of the present application is shown;

[0056] Figure 2 A schematic diagram of a workflow of a subtraction model provided in an embodiment of the present application is shown;

[0057] Figure 3 A schematic diagram of the workflow of another subtraction model provided in an embodiment of the present application is shown;

[0058] Figure 4 A schematic diagram of a workflow of another subtraction model provided in an embodiment of the present application is shown;

[0059] Figure 5 A schematic structural diagram of a subtraction image generating device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0060] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

[0061] In this embodiment, a method for generating a subtraction image is provided, such as Figure 1 As shown, including:

[0062] Step S1, obtaining a target angiography frame image;

[0063] Step S2, inputting the target angiography frame image into a trained subtraction model to obtain a target subtraction image corresponding to the target angiography frame image, wherein the subtraction model is trained by a first sample angiography frame image and its corresponding first sample subtraction reference image.

[0064] In the above embodiment, when performing angiography on a patient, the image is collected by a digital subtraction angiography (DSA) device. Usually, during angiography, the contrast agent is injected after a delay of several frames, and the image without contrast agent is used as a mask, i.e., a non-contrast frame image, and the frame image containing contrast agent is used as an angiography image, i.e., an angiography frame image. In a specific application scenario, a target angiography frame image is obtained, and the image is input into a trained subtraction model to obtain a corresponding target subtraction image, wherein the subtraction model is pre-trained using a first sample angiography frame image and a corresponding subtraction image (i.e., a first sample subtraction reference image).

[0065] Optionally, step S1 may specifically include: obtaining the angiography frame image to be subtracted and the corresponding non-angiography frame image, and calculating the target feature difference value between the angiography frame image to be subtracted and the non-angiography frame image; when the target feature difference value is greater than a preset difference threshold, using the angiography frame image to be subtracted as the target angiography frame image.

[0066] In the above embodiment, if the target feature difference value is greater than or equal to the preset difference threshold, it means that the degree of motion artifact is large. In this case, due to the large degree of displacement of the tissue structure, the way of registering the mask may be difficult to ensure accuracy. For example, the registration may cause the tissue structure image to be unnatural. In this scenario, the subtraction image generation model (referred to as the subtraction model) can be pre-trained, and the target angiography frame image can be input into the trained subtraction model to directly generate a subtraction image, thereby avoiding the problem of poor subtraction image effect due to the difficulty in ensuring the registration effect due to the large motion artifact, and improving the accuracy of the subtraction image.

[0067] Optionally, the above-mentioned "calculating the target feature difference value between the contrast frame image to be subtracted and the non-contrast frame image" may specifically include: determining a first image feature corresponding to the non-contrast frame image and a second image feature corresponding to the contrast frame image to be subtracted, wherein the first image feature includes a first grayscale feature of the non-contrast frame image, and the second image feature includes a second grayscale feature of the contrast frame image to be subtracted; based on the first grayscale feature and the second grayscale feature, calculating the grayscale feature difference value between the non-contrast frame image and the contrast frame image to be subtracted; wherein the target feature difference value includes the grayscale feature difference value, and the preset difference threshold includes a preset grayscale difference threshold, and the preset grayscale difference threshold has a value range of [0.01, 0.2], and the unit is percentage, that is, the grayscale difference threshold is the ratio of the difference between the grayscale mean of the non-contrast frame image and the grayscale mean of the contrast frame image to be subtracted to the grayscale mean of the non-contrast frame image.

[0068] In another example, the above-mentioned "calculating the target feature difference value between the contrast frame image to be subtracted and the non-contrast frame image" may specifically include: determining a first image feature corresponding to the non-contrast frame image and a second image feature corresponding to the contrast frame image to be subtracted, wherein the first image feature includes a first structural edge feature of the non-contrast frame image, and the second image feature includes a second structural edge feature of the contrast frame image to be subtracted; based on the first structural edge feature and the second structural edge feature, calculating the structural feature difference value between the non-contrast frame image and the contrast frame image to be subtracted; wherein the target feature difference value includes the structural feature difference value, and the preset difference threshold includes a preset structural difference Threshold, the preset structural difference threshold is a grayscale threshold, and the value range is [0.01, 0.4], the unit is percentage, that is, the structural difference threshold is the ratio of the difference between the grayscale mean of the edge feature of the non-contrast frame image and the grayscale mean of the edge feature of the contrast frame image to be subtracted to the grayscale mean of the edge feature of the non-contrast frame image, or the preset structural difference threshold is a position threshold, and the value range is [20, 200] pixels, that is, the structural difference threshold is the distance between the pixel position of the edge feature of the non-contrast frame image and the pixel position of the edge feature of the contrast frame image to be subtracted. The deformation field of the contrast frame image to be subtracted and the non-contrast frame image can be calculated by using the optical flow method or the like, and the deformation degree of the image is calculated using the displacement of the deformation field.

[0069] In the above embodiment, the image features of the mask and the angiogram are calculated respectively, including but not limited to the grayscale features of the image, the structural edge features, etc. The grayscale features are specifically the grayscale value features of each point in the image, and the structural edge features specifically include the position features of the edges of the anatomical structures contained in the image. Then, the feature difference between the mask and the angiogram is calculated to determine the degree of motion artifacts, wherein the feature difference is reflected by the grayscale feature difference value and the structural feature difference value, the grayscale feature difference value can specifically be the grayscale mean square error between each pixel point of the mask and the angiogram, and the structural feature difference value can specifically be the difference of the structural edge features, etc.

[0070] Compared with the registration method based on spatiotemporal information in the prior art, the embodiment of the present application can realize real-time registration of images, especially for the problem that the degree of displacement of tissue structure is large and the registration easily leads to unnatural tissue structure image. The target angiography frame image is input into the trained subtraction model to directly generate a subtraction image, thereby avoiding the problem of poor subtraction image effect due to the difficulty in ensuring the registration effect due to excessive motion artifacts, thereby improving the accuracy of the subtraction image.

[0071] In this embodiment, a subtraction model generation method is also provided, the method comprising:

[0072] Step 101, obtaining a first sample image, wherein the first sample image includes at least one first sample non-contrast frame image and at least one first sample contrast frame image, and a feature difference value between the first sample non-contrast frame image and the first sample contrast frame image is less than a preset sample threshold;

[0073] Step 102, subtracting any first sample contrast frame image from any first sample non-contrast frame image to obtain a first sample subtraction reference image, and forming a first sample image pair with the any first sample contrast frame image and the corresponding first sample subtraction reference image;

[0074] Step 103: Use the first sample image pair to train the subtraction model to be trained.

[0075] In an embodiment of the present application, first, a first sample image is obtained, wherein the first sample image can be an image collected by a digital subtraction angiography (DSA) device when performing angiography on a sample object. Usually, during angiography, the contrast agent will be injected after a delay of several frames, and the image without contrast agent is used as a mask, i.e., a non-contrast frame image, and the frame image containing contrast agent is used as an angiography film, i.e., an angiography frame image. In order to quickly obtain a subtraction image corresponding to the sample image, a sequence with basically no motion artifacts can be selected from the images collected by the DSA device as a training data set. Specifically, multiple first sample non-contrast frame images and multiple first sample angiography frame images corresponding to the first sample object can be obtained, and the sample images are screened according to the feature difference value to ensure that the feature difference value between any first sample non-contrast frame image and any first sample angiography frame image is less than a preset sample threshold. In a specific application scenario, since there is basically no motion artifact, step 101 can only select one first sample non-contrast frame image to reduce the amount of subsequent calculations.

[0076] Then, each first sample contrast frame image and first sample non-contrast frame image is subtracted to obtain a first sample subtraction reference image, and the first sample subtraction reference image is paired with the corresponding first sample contrast frame image to form a first sample image pair as a model training sample.

[0077] Finally, a subtraction model is constructed and trained using the first sample image pair. Specifically, the first sample angiography frame image can be used as the model input, and the first sample subtraction reference image can be used as the label data. Through the loss function, based on the feature difference value of the first sample subtraction reference image and the subtraction image output by the subtraction model, iteration is performed to achieve the training of the model. Among them, the subtraction model can select a spatial pyramid network containing densely connected convolutions. The trained subtraction model can predict the corresponding subtraction image based on a frame of angiography film, that is, the angiography film is input into the subtraction model, and the subtraction model can output the corresponding subtraction image. Using this model, there is no need to generate the subtraction image through mask registration, which avoids the problem of poor subtraction image effect due to the difficulty in ensuring the registration effect due to excessive motion artifacts, improves the accuracy of the subtraction image, and only one frame of angiography film is needed, which can realize real-time generation of subtraction images and improve the generation efficiency of subtraction images.

[0078] In the embodiment of the present application, optionally, step 103 may specifically include:

[0079] Step 103-1, input the first sample angiography frame image in any first sample image pair into the subtraction model to be trained to obtain a first sample subtraction prediction image, and iterate the subtraction model using a loss function based on the feature difference value between the first sample subtraction reference image and the first sample subtraction prediction image, until the loss function of the obtained subtraction model meets the preset subtraction condition, and the training of the subtraction model is completed.

[0080] In the above embodiment, if Figure 2 As shown, during the training process, for any first sample image pair, the first sample angiography frame image is input into the subtraction model to be trained to obtain the first sample subtraction prediction image, and then the feature difference value between the first sample subtraction prediction image and the first sample subtraction reference image in the first sample image pair is calculated, and the value is used as the input value of the loss function. The obtained loss function is used to iterate the model network parameters until the loss function stabilizes within a certain range or the number of iterations reaches a preset number, and the training of the subtraction model is completed.

[0081] In addition, in order to solve the problem of unnatural graphics caused by registration, and the problem that blood vessels and other areas are difficult to register because there are no blood vessels and other tissue structures in the mask but there are blood vessels and other tissue structures in the angiography, resulting in different image information (this problem can also be understood as the tissue structure in the angiography is blocked by the structure after the angiography, the image information is lost, and this part cannot be matched through image information). The registration loss function of the model can be determined by the regularization term + tissue deweighting perception loss. The loss function may also select a vascular downweighting perception loss, and the specific calculation method may be: using a pre-trained vascular extraction model such as a vascular segmentation network, perform vascular segmentation on the input first sample angiography frame image to generate a segmentation result, i.e., a first sample vascular segmentation image; then, using the first sample vascular segmentation image to downweight the first sample subtraction prediction image, specifically, the pixels containing blood vessels in the first sample vascular segmentation image may be recorded, and using these pixel information to downweight the pixels corresponding to the first sample subtraction prediction image (i.e., multiplying by a coefficient of <1), thereby reducing the impact caused by blood vessels; finally, calculating the feature difference value between the downweighted first sample subtraction prediction image and the first sample subtraction reference image, and using the value as the downweighting perception loss function of the model.

[0082] In the embodiment of the present application, optionally, step 103 may specifically include:

[0083] Step 103-2, acquiring a plurality of second sample image pairs, wherein any second sample image pair comprises a second sample angiography frame image and a second sample subtraction reference image, and the second sample image pair corresponds to a second part, wherein the first part and the second part correspond to images at different positions of the sample;

[0084] Step 103-3, training a preset neural network model according to the second sample image pair to obtain an auxiliary model;

[0085] Step 103 - 4 , establishing the subtraction model to be trained based on the auxiliary model and the preset neural network model, and using the first sample image pair to train the subtraction model to be trained.

[0086] In the above embodiment, in order to solve the problem that there are few motion artifact-free images of certain parts (such as the heart) in the DSA image sequence and insufficient training samples, the embodiment of the present application also provides a solution based on transfer learning that can be used in the design of the subtraction model. In a specific application scenario, some motion artifact-free samples of different parts, i.e., second sample image pairs, such as samples of the head and abdomen, can be used first, and the preset neural network model training can be completed in the manner of steps 101 to 103 as described above to obtain an auxiliary model, thereby constructing a subtraction model based on the auxiliary model and the initial preset neural network model.

[0087] Optionally, step 103-4 may specifically include: obtaining a preset migration layer in the auxiliary model as a migration model; inputting the first sample contrast frame image in the first sample image pair into the migration model to obtain migration data corresponding to the first sample contrast frame image, and constructing a first sample migration data pair including the migration data and the corresponding first sample subtraction reference image; obtaining a preset migrated layer in the initial preset neural network model as a migrated model, and training the migrated model using the second sample migration data pair; splicing the migration model and the trained migrated model to obtain the subtraction model.

[0088] Among them, a part of the front network layer is extracted from the trained neural network model, i.e., the auxiliary model, and the extracted network structure is used as the migration model, such as Figure 3 As shown, the network layer to be extracted can be agreed upon, that is, a preset migration layer is set, and then the first sample contrast frame image in the first sample image pair is input into the migration model to obtain the corresponding migration data, that is, the feature matrix of the first sample contrast frame image is generated using the intermediate layer (that is, the preset migration layer) of the pre-trained neural network model. Thus, the migrated part corresponding to the preset neural network model is trained using the migration data as a training sample. Specifically, the preset migrated layer in the preset neural network model can be obtained as the migrated model, so that the migrated model is trained using the migration data and the first sample subtraction reference image corresponding to the migration data, thereby reducing the number of layers of model training and the amount of network parameter data that needs to be determined. In this way, the network parameters of the migration model do not need to be changed, and only the parameters of the remaining network layers (that is, the preset migrated layers) need to be trained. Finally, based on the same network structure, the migration model and the migrated model can be spliced ​​together to obtain the final subtraction model.

[0089] In an embodiment of the present application, optionally, step 103-4 may specifically include: cascading the auxiliary model to the preset neural network model to obtain the subtraction model to be trained, and using the first sample image to train the subtraction model to be trained.

[0090] In the above embodiment, if Figure 4 As shown, after determining the second sample image pair, the preset neural network model is trained using the second sample image pair, and then the trained preset neural network model is cascaded to the initial preset neural network model to determine the subtraction model to be trained. Specifically, the preset neural network model can be divided into two parts according to the network layer, namely subtraction network 1 and subtraction network 2, which are cascaded to the middle position between the two, and then the first sample image pair is continued to be used for model training.

[0091] In an embodiment of the present application, optionally, step 101 may specifically include: pairing any first sample contrast frame image with any first sample non-contrast frame image to obtain an intermediate image pair; subtracting the first sample contrast frame image from the first sample non-contrast frame image in any intermediate image pair to obtain the first sample subtraction reference image, and using the first sample contrast frame image and the first sample subtraction reference image corresponding to any intermediate image pair as the first sample image pair.

[0092] In the above embodiment, the sample images may be paired first to form intermediate image pairs, each of which includes a first sample non-contrast frame image and a first sample contrast frame image. Then, for each intermediate image pair, the first sample contrast frame image and the first sample non-contrast frame image are subtracted to obtain a first sample subtraction reference image, and the first sample subtraction reference image and the first sample contrast frame image are used as a first sample image pair, so as to improve the pairing efficiency of the sample image pairs.

[0093] In an embodiment of the present application, optionally, a method for calculating a feature difference value between the first sample non-contrast frame image and the first sample contrast frame image includes:

[0094] Determine a first image feature corresponding to the first sample non-contrast frame image and a second image feature corresponding to the first sample contrast frame image, wherein the first image feature includes a first grayscale feature or a first structural edge feature of the first sample non-contrast frame image, and the second image feature includes a second grayscale feature or a second structural edge feature of the first sample contrast frame image; calculate a grayscale feature difference value between the first sample non-contrast frame image and the first sample contrast frame image based on the first grayscale feature and the second grayscale feature, or calculate a structural feature difference value between the first sample non-contrast frame image and the first sample contrast frame image based on the first structural edge feature and the second structural edge feature, wherein the feature difference value includes the grayscale feature difference value or the structural feature difference value.

[0095] In the above embodiment, the image features of the mask and the angiogram are calculated respectively, including but not limited to the grayscale features of the image, the structural edge features, etc. The grayscale features are specifically the grayscale value features of each point in the image, and the structural edge features specifically include the position features of the edges of the anatomical structures contained in the image. Then, the feature difference between the mask and the angiogram is calculated to determine the degree of motion artifacts, wherein the feature difference is reflected by the grayscale feature difference value and the structural feature difference value, the grayscale feature difference value can specifically be the grayscale mean square error between each pixel point of the mask and the angiogram, and the structural feature difference value can specifically be the difference of the structural edge features, etc. In addition, the preset sample threshold in step 101 can be the same as the preset difference threshold described above.

[0096] Further, as Figure 1 The specific implementation of the method, the embodiment of the present application provides a subtraction image generation device, such as Figure 5 As shown, the device comprises:

[0097] A target image acquisition module is used to acquire a target angiography frame image;

[0098] The subtraction image generation module is used to input the target angiography frame image into a trained subtraction model to obtain a target subtraction image corresponding to the target angiography frame image, wherein the subtraction model is trained by a first sample angiography frame image and its corresponding first sample subtraction reference image.

[0099] Optionally, the target image acquisition module is specifically used to:

[0100] Acquire a contrast-enhanced frame image to be subtracted and a corresponding non-contrast-enhanced frame image, and calculate a target feature difference value between the contrast-enhanced frame image to be subtracted and the non-contrast-enhanced frame image;

[0101] When the target feature difference value is greater than a preset difference threshold, the angiography frame image to be subtracted is used as the target angiography frame image.

[0102] Optionally, the target image acquisition module is further used for:

[0103] Specifically used for: determining a first image feature corresponding to the non-contrast frame image and a second image feature corresponding to the target contrast frame image, wherein the first image feature includes a first grayscale feature of the non-contrast frame image, and the second image feature includes a second grayscale feature of the contrast frame image to be subtracted; calculating a grayscale feature difference value between the non-contrast frame image and the contrast frame image to be subtracted based on the first grayscale feature and the second grayscale feature; wherein the target feature difference value includes the grayscale feature difference value, the preset difference threshold includes a preset grayscale difference threshold, and the preset grayscale difference threshold has a value range of [0.01, 0.2], in percentage; or,

[0104] Used to determine the first image feature corresponding to the non-contrast frame image and the second image feature corresponding to the contrast frame image to be subtracted, wherein the first image feature includes the first structural edge feature of the non-contrast frame image, and the second image feature includes the second structural edge feature of the contrast frame image to be subtracted; according to the first structural edge feature and the second structural edge feature, the structural feature difference value between the non-contrast frame image and the contrast frame image to be subtracted is calculated; wherein the target feature difference value includes the structural feature difference value, the preset difference threshold includes a preset structural difference threshold, the preset structural difference threshold is a gray threshold, and the value range is [0.01, 0.4], or the preset structural difference threshold is a position threshold, and the value range is [20, 200] pixels. .

[0105] Optionally, the device further comprises:

[0106] A model training module is used to obtain a first sample image, wherein the first sample image includes at least one first sample non-contrast frame image and at least one first sample contrast frame image, and the feature difference value between the first sample non-contrast frame image and the first sample contrast frame image is less than a preset sample threshold; subtract any first sample contrast frame image from any first sample non-contrast frame image to obtain a first sample subtraction reference image, and form a first sample image pair with any first sample contrast frame image and the corresponding first sample subtraction reference image; and use the first sample image pair to train the subtraction model to be trained.

[0107] Optionally, the model training module is specifically used to: input the first sample angiography frame image in any first sample image pair into the subtraction model to be trained to obtain a first sample subtraction prediction image, and iterate the subtraction model using a loss function based on the feature difference value between the first sample subtraction reference image and the first sample subtraction prediction image, until the loss function of the obtained subtraction model meets the preset subtraction condition, thereby completing the training of the subtraction model.

[0108] Optionally, the model training module is also used to:

[0109] Acquire a plurality of second sample image pairs, wherein any second sample image pair comprises a second sample contrast frame image and a second sample subtraction reference image, and the second sample image pair corresponds to a second part, wherein the first part and the second part correspond to images of different positions of the sample;

[0110] Training a preset neural network model according to the second sample image pair to obtain an auxiliary model;

[0111] The subtraction model to be trained is established based on the auxiliary model and the preset neural network model, and the subtraction model to be trained is trained using the first sample image pair.

[0112] Optionally, the model training module is also used to:

[0113] Acquire a preset migration layer in the auxiliary model as a migration model;

[0114] Inputting the first sample contrast frame image in the first sample image pair into the migration model to obtain migration data corresponding to the first sample contrast frame image, and constructing a first sample migration data pair including the migration data and the corresponding first sample subtraction reference image;

[0115] Acquire a preset migrated layer in the initial preset neural network model as a migrated model, and train the migrated model using the second sample migration data;

[0116] The migration model and the trained migrated model are spliced ​​to obtain the subtraction model.

[0117] It should be noted that for other corresponding descriptions of the functional units involved in the subtraction image generation device provided in the embodiment of the present application, reference can be made to Figures 1 to 4 The corresponding description in the method will not be repeated here.

[0118] Based on the above Figures 1 to 4 The method shown in the embodiment of the present application is accordingly provided with a storage medium on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned Figures 1 to 4 The method shown.

[0119] Based on this understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each implementation scenario of the present application.

[0120] Based on the above Figures 1 to 4 The method shown in the figure, and the virtual device embodiment, in order to achieve the above purpose, the embodiment of the present application also provides a computer device, which can be a personal computer, a server, a network device, etc., and the computer device includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to achieve the above Figures 1 to 4 The method shown.

[0121] Optionally, the computer device may further include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a WI-FI module, etc. The user interface may include a display, an input unit such as a keyboard, etc., and the optional user interface may also include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (such as a Bluetooth interface, a WI-FI interface), etc.

[0122] Those skilled in the art will appreciate that the computer device structure provided in this embodiment does not limit the computer device, and may include more or fewer components, or a combination of certain components, or different component arrangements.

[0123] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages and saves the hardware and software resources of the computer device, and supports the operation of information processing programs and other software and / or programs. The network communication module is used to realize communication between the components inside the storage medium, and communication with other hardware and software in the physical device.

[0124] Through the description of the above implementation methods, technical personnel in this field can clearly understand that the present application can be implemented by means of software plus the necessary general hardware platform, and can also realize real-time image registration through hardware, especially for the problem that the degree of tissue structure displacement is large and the registration easily leads to unnatural tissue structure images. The target angiography frame image is input into the trained subtraction model to directly generate a subtraction image, so as to avoid the problem of poor subtraction image effect due to the difficulty in ensuring the registration effect due to excessive motion artifacts, thereby improving the accuracy of the subtraction image.

[0125] Those skilled in the art will appreciate that the accompanying drawings are only schematic diagrams of a preferred implementation scenario, and the modules or processes in the accompanying drawings are not necessarily necessary for implementing the present application. Those skilled in the art will appreciate that the modules in the devices in the implementation scenario can be distributed in the devices of the implementation scenario according to the description of the implementation scenario, or can be changed accordingly and located in one or more devices different from the present implementation scenario. The modules of the above-mentioned implementation scenario can be combined into one module, or can be further split into multiple submodules.

[0126] The above serial numbers of this application are only for description and do not represent the advantages and disadvantages of the implementation scenarios. The above disclosure is only a few specific implementation scenarios of this application, but this application is not limited to them, and any changes that can be thought of by technicians in this field should fall within the scope of protection of this application.

Claims

1. A method for generating a subtraction image, characterized in that: include: Acquire a target angiography frame image; The target angiography frame image is input into a trained subtraction model to obtain a target subtraction image corresponding to the target angiography frame image, wherein the subtraction model is trained by a first sample angiography frame image and its corresponding first sample subtraction reference image, and the loss function of the subtraction model is calculated as follows: a pre-trained vascular segmentation network is used to perform vascular segmentation on the first sample angiography frame image to obtain a first sample vascular segmentation image; pixel information of pixels with blood vessels in the first sample vascular segmentation image is used to downweight the corresponding pixels in the first sample subtraction prediction image obtained by predicting the first sample angiography frame image by the subtraction model, and a feature difference value between the downweighted first sample subtraction prediction image and the first sample silhouette reference image is calculated.

2. The method according to claim 1, characterized in that The step of acquiring the target angiography frame image specifically includes: Acquire a contrast-enhanced frame image to be subtracted and a corresponding non-contrast-enhanced frame image, and calculate a target feature difference value between the contrast-enhanced frame image to be subtracted and the non-contrast-enhanced frame image; When the target feature difference value is greater than a preset difference threshold, the angiography frame image to be subtracted is used as the target angiography frame image.

3. The method according to claim 2, characterized in that The calculating of the target feature difference value between the contrast frame image to be subtracted and the non-contrast frame image specifically includes: Determine a first image feature corresponding to the non-contrast frame image and a second image feature corresponding to the target contrast frame image, wherein the first image feature includes a first grayscale feature of the non-contrast frame image, and the second image feature includes a second grayscale feature of the target contrast frame image; calculate a grayscale feature difference value between the non-contrast frame image and the target contrast frame image based on the first grayscale feature and the second grayscale feature; wherein the target feature difference value includes the grayscale feature difference value, the preset difference threshold includes a preset grayscale difference threshold, and the preset grayscale difference threshold has a value range of [0.01, 0.2], in percentage; or, Determine a first image feature corresponding to the non-contrast frame image and a second image feature corresponding to the target contrast frame image, wherein the first image feature includes a first structural edge feature of the non-contrast frame image, and the second image feature includes a second structural edge feature of the target contrast frame image; calculate a structural feature difference value between the non-contrast frame image and the target contrast frame image based on the first structural edge feature and the second structural edge feature; wherein the target feature difference value includes the structural feature difference value, and the preset difference threshold includes a preset structural difference threshold, and the preset structural difference threshold is a grayscale threshold with a value range of [0.01, 0.4], or the preset structural difference threshold is a position threshold with a value range of [20, 200] pixels.

4. The method according to claim 1, characterized in that: The training steps of the subtraction model include: Acquire a first sample image, wherein the first sample image includes at least one first sample non-contrast frame image and at least one first sample contrast frame image, and a feature difference value between the first sample non-contrast frame image and the first sample contrast frame image is less than a preset sample threshold; Subtracting any first sample contrast frame image from any first sample non-contrast frame image to obtain a first sample subtraction reference image, and forming a first sample image pair with the any first sample contrast frame image and the corresponding first sample subtraction reference image; The subtraction model to be trained is trained using the first sample image pair.

5. The method according to claim 4, characterized in that The using the first sample image pair to train the subtraction model to be trained specifically includes: The first sample angiography frame image in any first sample image pair is input into the subtraction model to be trained to obtain a first sample subtraction prediction image, and based on the feature difference value between the first sample subtraction reference image and the downweighted first sample subtraction prediction image, the subtraction model is iterated using a loss function until the loss function of the obtained subtraction model meets the preset subtraction condition, thereby completing the training of the subtraction model.

6. The method according to claim 5, characterized in that The using the first sample image pair to train the subtraction model to be trained specifically includes: Acquire a plurality of second sample image pairs, wherein any second sample image pair comprises a second sample angiography frame image and a second sample subtraction reference image, and the second sample image pair corresponds to a second part, wherein the first part and the second part correspond to images of different positions of the sample, and the first part is the position of the sample corresponding to the first sample image pair; Training a preset neural network model according to the second sample image pair to obtain an auxiliary model; The subtraction model to be trained is established based on the auxiliary model and the preset neural network model, and the subtraction model to be trained is trained using the first sample image pair.

7. The method according to claim 6, characterized in that The step of establishing the subtraction model to be trained based on the auxiliary model and the preset neural network model, and training the subtraction model to be trained using the first sample image pair, specifically includes: Acquire a preset migration layer in the auxiliary model as a migration model; Inputting the first sample contrast frame image in the first sample image pair into the migration model to obtain migration data corresponding to the first sample contrast frame image, and constructing a first sample migration data pair including the migration data and the corresponding first sample subtraction reference image; Acquire a preset migrated layer in the initial preset neural network model as a migrated model, and train the migrated model using the first sample migration data; The migration model and the trained migrated model are spliced ​​to obtain the subtraction model.

8. A subtraction image generating device, characterized in that: include: A target image acquisition module is used to acquire a target angiography frame image; A subtraction image generation module is used to input the target angiography frame image into a trained subtraction model to obtain a target subtraction image corresponding to the target angiography frame image, wherein the subtraction model is trained by a first sample angiography frame image and its corresponding first sample subtraction reference image, and the loss function of the subtraction model is calculated as follows: using a pre-trained vascular segmentation network to perform vascular segmentation on the first sample angiography frame image to obtain a first sample vascular segmentation image; using pixel information of pixels with blood vessels in the first sample vascular segmentation image, the corresponding pixels in the first sample subtraction prediction image obtained by predicting the first sample angiography frame image by the subtraction model are down-weighted, and the feature difference value between the down-weighted first sample subtraction prediction image and the first sample silhouette reference image is calculated.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

Images