Coronary CT image subtraction method, device, electronic device and storage medium

Abstract

The present application provides a method, device, electronic device, and storage medium for subtracting coronary CT images, including: obtaining an enhanced CT image and a plain CT image of a target patient; extracting a first feature point set of the enhanced CT image using a first feature point extraction model; extracting a second feature point set of the plain CT image using a second feature point extraction model; determining a target transformation matrix based on the first feature point set and the second feature point set; performing coordinate transformation processing based on the plain CT image and the target transformation matrix to obtain a transformed plain CT image; and obtaining a subtracted CT image by subtracting the grayscale values ​​of the corresponding pixels in the transformed plain CT image from the grayscale values ​​of all pixels in the enhanced CT image. The subtraction method provided by this solution can automatically obtain a subtracted CT image, and by accurately registering the enhanced CT image and the plain CT image, the accuracy of the subtracted CT image is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device, electronic device and storage medium for subtraction of coronary CT images. Background Art

[0002] Coronary heart disease (CHD) is known for its high morbidity and mortality rates, making it the leading cause of death both domestically and internationally. Therefore, early diagnosis and precise treatment of CHD are crucial for preventing the disease in the general population and prolonging life expectancy in those with the disease. Plain CT, a commonly used screening tool for CHD, is often used to calculate indicators such as the calcium score, while enhanced CT is used to establish three-dimensional coronary artery models and quantitatively and qualitatively observe plaques. In practice, enhanced CT often obscures the high-density shadows in areas of severe calcification or stents, making it difficult to visualize the lumen. These high-density shadows are less affected by contrast agents and therefore appear more consistent on enhanced and plain CT. Therefore, subtraction processing based on plain CT can be used to remove the stents and calcifications displayed on enhanced CT, thereby providing a clearer view of the lumen and enabling more accurate subsequent three-dimensional lumen modeling or functional calculations such as CT-FFR. Therefore, automatic subtraction processing based on these two images is a technical problem that needs to be urgently addressed. Summary of the Invention

[0003] In view of this, the purpose of this application is to provide a method, device, electronic device and storage medium for subtraction of coronary CT images, which can automatically obtain subtraction CT images and improve the accuracy of subtraction CT images by accurately aligning enhanced CT images and plain scan CT images.

[0004] The present invention provides a method for subtracting a coronary CT image. The method comprises:

[0005] Acquire an enhanced CT image and a plain CT image of a target patient; wherein the enhanced CT image and the plain CT image are both CT images of the coronary arteries of the target patient;

[0006] Inputting the enhanced CT image into a pre-trained first feature point extraction model to extract a first feature point set of the enhanced CT image;

[0007] Inputting the plain scan CT image into a pre-trained second feature point extraction model to extract a second feature point set of the plain scan CT image;

[0008] Determining a target transformation matrix between the first feature point set and the second feature point set through iterative calculation based on the first feature point set and the second feature point set;

[0009] performing coordinate transformation processing based on the plain scan CT image and the target transformation matrix to obtain a transformed plain scan CT image, so as to achieve image registration between the plain scan CT image and the enhanced CT image;

[0010] The grayscale values ​​of all pixels in the enhanced CT image are subtracted from the grayscale values ​​of corresponding pixels in the transformed plain scan CT image to obtain a subtraction CT image.

[0011] Optionally, the first feature point extraction model and the second feature point extraction model are constructed by the following steps:

[0012] Acquire a CT image training sample set; the CT image training sample set includes enhanced CT images and plain CT images of coronary arteries of multiple patients to be trained, and each CT image in the CT image training sample set has a corresponding feature point set label;

[0013] Determining the enhanced CT image to be trained as a first input feature of a first neural network, determining the feature point set label corresponding to the enhanced CT image to be trained as a first output feature of the first neural network, determining the plain scan CT image to be trained as a second input feature of a second neural network, and determining the feature point set label corresponding to the plain scan CT image to be trained as a second output feature of the second neural network;

[0014] Based on the first input feature, the first output feature, the second input feature, and the second output feature, performing alternating iterative training on the first neural network and the second neural network using a back gradient propagation algorithm; wherein, during the alternating iterative training, network parameters of the first neural network and the second neural network are shared;

[0015] When the first neural network and the second neural network reach a convergence state, the training is stopped to obtain a first feature point extraction model and a second feature point extraction model.

[0016] Optionally, determining a target transformation matrix between the first feature point set and the second feature point set by iterative calculation based on the first feature point set and the second feature point set includes:

[0017] Obtaining an initial transformation matrix; the initial transformation matrix includes multiple parameters corresponding to multiple transformation types, each parameter being set to a random initial value;

[0018] performing an initial transformation on the feature points in the second feature point set based on the initial transformation matrix to obtain a transformed second feature point set;

[0019] Setting an objective function based on the coordinates of the second feature point in the second feature point set after the transformation and the coordinates of the first feature point in the first feature point set; the objective function is used to determine the sum of the Euclidean distances between the second feature point and the first feature point after the transformation;

[0020] Based on the objective function, the random initial value corresponding to each parameter in the initial transformation matrix is ​​iteratively updated until the value of the objective function is less than a preset threshold, and the updating is stopped to obtain the target transformation matrix.

[0021] Optionally, the first neural network and the second neural network have the same initial network parameters.

[0022] Optionally, the alternately iteratively training the first neural network and the second neural network using a back gradient propagation algorithm based on the first input feature, the first output feature, the second input feature, and the second output feature includes:

[0023] In each iteration, a first loss function is determined based on the first input feature and the first output feature, and a second loss function is determined based on the second input feature and the second output feature;

[0024] Alternately selecting the first loss function or the second loss function as a target loss function, and determining a target neural network corresponding to the selected target loss function; wherein the target neural network corresponding to the first loss function is a first neural network, and the target neural network corresponding to the second loss function is a second neural network;

[0025] Based on the target loss function, the network parameters of the target neural network are updated using the back gradient propagation algorithm, and the network parameters of the other neural network are updated at the same time to make the network parameters of the two neural networks consistent.

[0026] Optionally, the transformation type includes at least translation transformation, rotation transformation, perspective transformation and scaling transformation.

[0027] The present application also provides a coronary CT image subtraction device, the subtraction device comprising:

[0028] An acquisition module, configured to acquire an enhanced CT image and a plain CT image of a target patient; the enhanced CT image and the plain CT image are both CT images of the coronary arteries of the target patient;

[0029] A first extraction module is configured to input the enhanced CT image into a pre-trained first feature point extraction model to extract a first feature point set of the enhanced CT image;

[0030] A second extraction module is configured to input the plain scan CT image into a pre-trained second feature point extraction model to extract a second feature point set of the plain scan CT image;

[0031] a matrix determination module, configured to determine, based on the first feature point set and the second feature point set, a target transformation matrix between the first feature point set and the second feature point set through iterative calculation;

[0032] a registration module, configured to perform coordinate transformation processing based on the plain scan CT image and the target transformation matrix to obtain a transformed plain scan CT image, so as to achieve image registration between the plain scan CT image and the enhanced CT image;

[0033] The subtraction module is configured to subtract the grayscale values ​​of corresponding pixels in the transformed plain scan CT image from the grayscale values ​​of all pixels in the enhanced CT image to obtain a subtracted CT image.

[0034] Optionally, the subtraction device further includes a model building module, and the model building module is used to:

[0035] Acquire a CT image training sample set; the CT image training sample set includes enhanced CT images and plain CT images of coronary arteries of multiple patients to be trained, and each CT image in the CT image training sample set has a corresponding feature point set label;

[0036] Determining the enhanced CT image to be trained as a first input feature of a first neural network, determining the feature point set label corresponding to the enhanced CT image to be trained as a first output feature of the first neural network, determining the plain scan CT image to be trained as a second input feature of a second neural network, and determining the feature point set label corresponding to the plain scan CT image to be trained as a second output feature of the second neural network;

[0037] Based on the first input feature, the first output feature, the second input feature, and the second output feature, performing alternating iterative training on the first neural network and the second neural network using a back gradient propagation algorithm; wherein, during the alternating iterative training, network parameters of the first neural network and the second neural network are shared;

[0038] When the first neural network and the second neural network reach a convergence state, the training is stopped to obtain a first feature point extraction model and a second feature point extraction model.

[0039] Optionally, when the matrix determination module is used to determine the target transformation matrix between the first feature point set and the second feature point set through iterative calculation based on the first feature point set and the second feature point set, the matrix determination module is used to:

[0040] Obtaining an initial transformation matrix; the initial transformation matrix includes multiple parameters corresponding to multiple transformation types, each parameter being set to a random initial value;

[0041] performing an initial transformation on the feature points in the second feature point set based on the initial transformation matrix to obtain a transformed second feature point set;

[0042] Setting an objective function based on the coordinates of the second feature point in the second feature point set after the transformation and the coordinates of the first feature point in the first feature point set; the objective function is used to determine the sum of the Euclidean distances between the second feature point and the first feature point after the transformation;

[0043] Based on the objective function, the random initial value corresponding to each parameter in the initial transformation matrix is ​​iteratively updated until the value of the objective function is less than a preset threshold, and the updating is stopped to obtain the target transformation matrix.

[0044] Optionally, the first neural network and the second neural network have the same initial network parameters.

[0045] Optionally, when the model construction module is used to perform alternating iterative training on the first neural network and the second neural network using a back gradient propagation algorithm based on the first input feature, the first output feature, the second input feature, and the second output feature, the model construction module is used to:

[0046] In each iteration, a first loss function is determined based on the first input feature and the first output feature, and a second loss function is determined based on the second input feature and the second output feature;

[0047] Alternately selecting the first loss function or the second loss function as a target loss function, and determining a target neural network corresponding to the selected target loss function; wherein the target neural network corresponding to the first loss function is a first neural network, and the target neural network corresponding to the second loss function is a second neural network;

[0048] Based on the target loss function, the network parameters of the target neural network are updated using the back gradient propagation algorithm, and the network parameters of the other neural network are updated at the same time to make the network parameters of the two neural networks consistent.

[0049] Optionally, the transformation type includes at least translation transformation, rotation transformation, perspective transformation and scaling transformation.

[0050] An embodiment of the present application also provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the subtraction method as described above are performed.

[0051] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned subtraction method are executed.

[0052] An embodiment of the present application provides a coronary CT image subtraction method, device, electronic device, and storage medium. The subtraction method includes: acquiring an enhanced CT image and a plain scan CT image of a target patient; the enhanced CT image and the plain scan CT image are both CT images of the coronary arteries of the target patient; inputting the enhanced CT image into a pre-trained first feature point extraction model to extract a first feature point set of the enhanced CT image; inputting the plain scan CT image into a pre-trained second feature point extraction model to extract a second feature point set of the plain scan CT image; determining a target transformation matrix between the first feature point set and the second feature point set through iterative calculation based on the first feature point set and the second feature point set; performing coordinate transformation processing based on the plain scan CT image and the target transformation matrix to obtain a transformed plain scan CT image to achieve image registration between the plain scan CT image and the enhanced CT image; and subtracting the grayscale values ​​of corresponding pixels in the transformed plain scan CT image from the grayscale values ​​of all pixels in the enhanced CT image to obtain a subtracted CT image.

[0053] Thus, this application extracts feature points from both the enhanced and plain CT images based on machine learning, pairs these feature points together to form point pairs, and constructs a target transformation matrix from these pairs. This transforms the plain CT image into the enhanced CT image space using the target transformation matrix, and finally subtracts the grayscale values ​​at corresponding locations in the enhanced and plain CT images, completing the closed-loop image subtraction task. Furthermore, this solution is fully automated, requiring no human intervention.

[0054] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0056] Figure 1 A flowchart of a coronary CT image subtraction method provided in an embodiment of the present application;

[0057] Figure 2 Schematic diagram of characteristic points of the heart region provided for this application;

[0058] Figure 3 A schematic diagram of the alternating iterative training neural network provided in this application;

[0059] Figure 4 A schematic diagram of a subtraction CT image provided in this application;

[0060] Figure 5 This is one of the structural schematic diagrams of a coronary CT image subtraction device provided in an embodiment of the present application;

[0061] Figure 6 This is a second structural schematic diagram of a coronary CT image subtraction device provided in an embodiment of the present application;

[0062] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application for which protection is claimed, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, each other embodiment obtained by those skilled in the art without making creative work falls within the scope of protection of the present application.

[0064] Coronary heart disease is known for its high morbidity and mortality rates, and is considered the number one killer both domestically and internationally. Therefore, early diagnosis and precise treatment of coronary heart disease play a vital role in preventing the disease in the general population and prolonging the lifespan of those with the disease. Plain CT, as a commonly used screening method for coronary heart disease, is often used to calculate indicators such as calcification scores, while enhanced CT is used to establish a three-dimensional model of the coronary arteries and for quantitative and qualitative observation of plaques. In actual practice, enhanced CT often makes it difficult to observe the lumen in areas with severe calcification or stents due to the high-density shadows. The above-mentioned high-density shadows are less affected by contrast agents, and therefore appear more consistent on enhanced and plain CT. Therefore, based on plain CT, the stents and calcifications displayed on the enhanced CT can be removed through subtraction processing, thereby making the lumen more clearly displayed, and subsequently making subsequent three-dimensional lumen modeling or functional calculations such as CT-FFR more accurate.

[0065] The existing subtraction processing methods based on enhanced CT and plain CT usually include the following two methods: (1) manually selecting some feature points on the coronary enhanced and plain CT, and matching them one by one. Based on these feature points, a spatial transformation matrix is ​​constructed, and the plain CT is transformed to the enhanced CT space through the transformation matrix. Finally, the grayscale values ​​of the corresponding positions of the enhanced CT and the plain CT are subtracted to complete the subtraction process. (2) There is no need to manually select feature points. Global matching is performed based on global points, and a global spatial transformation matrix is ​​constructed. The plain CT is transformed to the enhanced CT space through the transformation matrix. Finally, the grayscale values ​​of the corresponding positions of the enhanced CT and the plain CT are subtracted to complete the subtraction process.

[0066] However, technical means (1) has the following disadvantages: it requires manual selection of feature points, which is time-consuming and labor-intensive, increases the burden on medical staff, and is prone to inaccurate feature point selection due to human factors such as fatigue, thus affecting the final subtraction effect. Technical means (2) has the following disadvantages: although it does not require manual feature point selection, this global position-based solution does not focus on the heart area. Therefore, even if the overall image is well registered, the heart area that we are most concerned about is not well registered, resulting in deviations in the final subtraction.

[0067] Based on this, an embodiment of the present application provides a subtraction method for coronary CT images, which fully automatically completes the selection of feature points, the construction of the target transformation matrix, and the generation of subtraction CT images, thereby saving a lot of manpower and time, reducing the burden on medical staff, and improving the accuracy of subtraction CT images.

[0068] See also Figure 1 , Figure 1 This is a flow chart of a method for subtracting coronary CT images provided in an embodiment of the present application. Figure 1As shown in , the subtraction method provided in the embodiment of the present application includes:

[0069] S101 : Acquire an enhanced CT image and a plain CT image of a target patient.

[0070] Here, the enhanced CT image and the plain CT image are both CT images of the coronary arteries of the target patient.

[0071] The biggest difference between enhanced CT images and plain CT images is whether contrast agent is used during the acquisition. Plain CT images, which are acquired without contrast agent, are often used for initial screening of coronary artery disease and calculation of calcium scores. Enhanced CT images, on the other hand, are acquired with angiographic contrast agent, allowing for angiographic visualization of the blood vessels. They share the same observational similarity: both can reveal high-density structures such as calcifications and stents. However, they differ in that enhanced CT images provide a clearer view of the vascular lumen, whereas plain CT images struggle to visualize blood vessels.

[0072] S102: Input the enhanced CT image into a pre-trained first feature point extraction model to extract a first feature point set of the enhanced CT image.

[0073] Here, the first feature point extraction model is used to extract feature points from the input enhanced CT image, and can extract multiple first feature points from the enhanced CT image, and form a first feature point set based on the extracted multiple feature points.

[0074] When extracting the first specific points, the predicted labels and spatial coordinates of each first specific point are determined simultaneously. The predicted labels are used to match the feature points subsequently extracted based on the plain scan CT image.

[0075] The first feature point extracted is the key feature point of the heart region in the enhanced CT image. It should be noted that there are many feature points in the heart region that are significant in both enhanced and plain CT images, such as the junction of the aorta and coronary arteries, the junction of the heart and spine, and the junction of the aorta and pulmonary artery.

[0076] For examples, see Figure 2 , Figure 2 This is a schematic diagram of the heart area feature points provided in this application. Figure 2 As shown in the figure, the dots on the figure are the feature points that need to be extracted.

[0077] S103: Input the plain scan CT image into a pre-trained second feature point extraction model to extract a second feature point set of the plain scan CT image.

[0078] Here, the second feature point extraction model is used to extract feature points from the input plain scan CT image, and can extract multiple second feature points from the plain scan CT image, and form a second feature point set based on the extracted multiple feature points.

[0079] When extracting the second specific points, a predicted label and spatial coordinates, etc., are simultaneously determined for each second specific point. The predicted label is used to match the feature points extracted from the enhanced CT image. The number of feature points included in the second feature point set is the same as the number of feature points included in the first feature point set, and each first feature point has a unique matching second feature point, specifically matching the predicted label.

[0080] In one embodiment provided in the present application, the first feature point extraction model and the second feature point extraction model are constructed by the following steps: obtaining a CT image training sample set; the CT image training sample set includes enhanced CT images to be trained and plain CT images to be trained of the coronary arteries of multiple patients, and each CT image in the CT image training sample set has a corresponding feature point set label; determining the enhanced CT image to be trained as a first input feature of a first neural network, determining the feature point set label corresponding to the enhanced CT image to be trained as a first output feature of the first neural network, determining the plain CT image to be trained as a second input feature of a second neural network, and determining the feature point set label corresponding to the plain CT image to be trained as a second output feature of the second neural network; based on the first input feature, the first output feature, the second input feature, and the second output feature, using a back gradient propagation algorithm to perform alternating iterative training on the first neural network and the second neural network; wherein, during the alternating iterative training, the network parameters of the first neural network and the second neural network are shared; when the first neural network and the second neural network reach a convergence state, the training is stopped to obtain the first feature point extraction model and the second feature point extraction model.

[0081] Here, the CT image training sample set is composed of multiple CT image pairs, each of which includes a to-be-trained enhanced CT image and a to-be-trained plain CT image of a patient's coronary arteries. Each to-be-trained enhanced CT image and to-be-trained plain CT image are annotated with feature points, i.e., each to-be-trained enhanced CT image and to-be-trained plain CT image has a corresponding feature point set label.

[0082] When annotating feature points of the to-be-trained enhanced CT image and the to-be-trained plain scan CT image, manual annotation can be used.

[0083] In another embodiment provided by the present application, the initial network parameters of the first neural network and the second neural network are the same.

[0084] Here, the first neural network and the second neural network are two sub-networks in a twin neural network.

[0085] It should be noted that a twin neural network refers to an artificial neural network (ANN) with two sub-networks, and the structures and parameter weights of these two sub-networks are the same. Generally speaking, the input of a twin neural network is two signals (one-dimensional or multi-dimensional) with a large degree of similarity but certain differences. The advantage of this network is that it can better measure the similarity of the two inputs. However, there are differences in the coronary artery area and similarities in most other areas between plain scan CT and enhanced CT. Therefore, the extraction of feature points of the two images is completed through multi-layer information interaction based on the twin neural network. Therefore, this scheme fully utilizes the advantages of the twin neural network in extracting the similarity features of the two input images, takes into account the differences between them, and adopts the intermediate multi-layer information interaction method to interact the difference features.

[0086] Among them, when determining the input features of the first neural network and the second neural network, each neural network only uses a unique type of image as the input feature. Here, the enhanced CT image is used as the input feature of the first neural network, and the plain scan CT is used as the input feature of the second neural network. In this way, the first input feature and the second input feature are determined, and the corresponding feature point set labels are used as the output features corresponding to the input features.

[0087] Here, after determining the input and output features of each neural network, model training can be performed. This solution uses a back-gradient propagation algorithm to iteratively train the two neural networks. During training, the network parameters of the first and second neural networks are shared. Network parameter sharing means that when the network parameters of one neural network are updated, the network parameters of the other neural network are also updated in the same manner to ensure that the network parameters of the two neural networks are identical.

[0088] In another embodiment provided in the present application, the first neural network and the second neural network are alternately iteratively trained based on the first input feature, the first output feature, the second input feature, and the second output feature using a back gradient propagation algorithm, including: in each iteration, a first loss function is determined based on the first input feature and the first output feature, and a second loss function is determined based on the second input feature and the second output feature; the first loss function or the second loss function is alternately selected as the target loss function, and a target neural network corresponding to the selected target loss function is determined; wherein the target neural network corresponding to the first loss function is the first neural network, and the target neural network corresponding to the second loss function is the second neural network; based on the target loss function, the network parameters of the target neural network are updated using a back gradient propagation algorithm, and the network parameters of the other neural network are simultaneously updated to make the network parameters of the two neural networks consistent.

[0089] Here, when determining the first loss function based on the first input feature and the first output feature, specifically, the first input feature is input into a first neural network to obtain a first prediction result of the first input feature, and the first prediction result is compared and calculated with the first output feature to obtain the first loss function. Determining the second loss function based on the second input feature and the second output feature specifically involves inputting the second input feature into a second neural network to obtain a second prediction result of the second input feature, and the second prediction result is compared and calculated with the second output feature to obtain the second loss function.

[0090] For examples, see Figure 3 , Figure 3 This is a schematic diagram of the alternating iterative training neural network provided in this application. Specifically, Figure 3As shown, the network parameters of the two neural networks are shared, that is, in one iterative training, the enhanced CT is first input into the upper neural network (first neural network), and the feature point prediction results are obtained and compared with the standard feature point results (feature point set labels) to obtain the loss function value (first loss function). The network parameters of the upper sub-network are updated through reverse gradient propagation, and the network parameters of the lower neural network (second neural network) are updated according to the updated network parameters of the upper neural network. After the update, the network parameters of the two neural networks are the same, wherein the network parameters are updated to make the obtained feature point prediction results closer to the standard feature point results; secondly, the plain scan CT is input into the lower neural network, and the feature point prediction results are obtained and compared with the standard feature point results to obtain the loss function value (second loss function). The network parameters of the lower neural network are updated through reverse gradient propagation, and the network parameters of the upper neural network (second neural network) are updated according to the updated network parameters of the lower neural network. After the update, the network parameters of the two neural networks are the same. After a large number of alternating iterative training of enhanced CT scans to be trained and plain scan CT scans to be trained, the loss function values ​​of the two neural networks no longer decrease, that is, the network converges, and the training is stopped, and the first feature point extraction model and the second feature point extraction model can be obtained.

[0091] In this way, after the training is completed, in actual applications, the enhanced CT and plain scan CT can be input in parallel into the upper and lower neural networks (the first neural network and the second neural network) of the twin neural network to complete the extraction of the two image feature points in parallel.

[0092] S104 : Determine a target transformation matrix between the first feature point set and the second feature point set through iterative calculation based on the first feature point set and the second feature point set.

[0093] It should be noted that, in general, after a plain CT scan, a contrast agent is injected and then an enhanced CT scan is taken. During this process, the patient's position does not change significantly, so the cardiac region in the enhanced and plain CT scans can be roughly considered unchanged. Therefore, image rigid body registration (also known as image rigid registration) can be used to register the plain CT scan to the spatial location of the enhanced CT scan. The essence of a rigid body transformation is the spatial transformation of coordinate points, that is, the spatial transformation of feature points on the two images. During this transformation process, the transformation matrix is ​​crucial in determining the transformation method, which necessitates the calculation of the target transformation matrix.

[0094] In one embodiment provided in the present application, the target transformation matrix between the first feature point set and the second feature point set is determined through iterative calculation based on the first feature point set and the second feature point set, including: obtaining an initial transformation matrix; the initial transformation matrix includes multiple parameters corresponding to multiple transformation types, and each parameter is set with a random initial value; based on the initial transformation matrix, the feature points in the second feature point set are initially transformed to obtain the transformed second feature point set; based on the coordinates of the second feature point in the transformed second feature point set and the coordinates of the first feature point in the first feature point set, an objective function is set; the objective function is used to determine the sum of the Euclidean distances between the transformed second feature point and the first feature point; based on the objective function, the random initial value corresponding to each parameter in the initial transformation matrix is ​​iteratively updated until the value of the objective function is less than a preset threshold, and the update is stopped to obtain the target transformation matrix.

[0095] Here, the transformation types include at least translation transformation, rotation transformation, perspective transformation and scaling transformation.

[0096] Among them, the coordinate conversion calculation formula corresponding to the transformation matrix is ​​as follows:

[0097]

[0098] As shown in the above calculation formula, there are four types of parameters in the transformation matrix, among which a 11 to a 33 represents the rotation parameter, t x to t z represents the translation parameter, v x to v z represents the perspective transformation parameter, and s represents the scaling factor. Essentially, the rotation matrix has three unknown parameters, representing the rotation angles along the x, y, and z axes. This adds up to a total of 10 unknown parameters. Determining the target transformation matrix involves determining the specific values ​​of these 10 unknown parameters.

[0099] If we consider the first feature point set of the enhanced CT image to be Q = {q1, q2, ...q n}, the second feature point set of the plain scan CT image is P = {p1, p2, ... p n}, first set the 10 unknown parameters of the initial transformation matrix to a random initial value, and transform all the feature points in P to the new coordinate point P'={p1', p2', ...p n '}, and set the objective function, which is used to determine the sum of the Euclidean distances between the second feature point and the first feature point after the transformation. The formula of the set objective function is as follows:

[0100]

[0101] The Euclidean distance between the feature points of the transformed plain CT image and the feature points on the enhanced CT is calculated by the above objective function. Then, this objective function is iteratively optimized so that the transformation matrix composed of 10 unknown parameters can make the value of the objective function less than the preset threshold, that is, the condition is met, and the final target transformation matrix is ​​obtained.

[0102] S105 , performing coordinate transformation processing based on the plain scan CT image and the target transformation matrix to obtain a transformed plain scan CT image, so as to achieve image registration between the plain scan CT image and the enhanced CT image.

[0103] Here, based on the plain scan CT image and the target transformation matrix, a transformed plain scan CT image is obtained. In fact, all pixels in the plain scan CT image are transformed to the same space as the enhanced CT image through the target transformation matrix determined above, thereby obtaining the transformed plain scan CT image.

[0104] Wherein, the image registration is image rigid registration.

[0105] It should be noted that image registration is the process of matching and superimposing two or more images acquired at different times, using different sensors (imaging devices) or under different conditions (weather, illumination, camera position and angle, etc.). It has been widely used in remote sensing data analysis, computer vision, image processing and other fields.

[0106] S106 , subtracting the grayscale values ​​of corresponding pixels in the transformed plain scan CT image from the grayscale values ​​of all pixels in the enhanced CT image to obtain a subtraction CT image.

[0107] Here, for each pixel on the enhanced CT image, the grayscale value of the pixel is subtracted from the grayscale value of the pixel at the corresponding position in the transformed plain scan CT image. When the grayscale values ​​of all pixels are subtracted, the resulting image is the subtraction CT image.

[0108] For examples, see Figure 4 As shown, Figure 4 Schematic diagram of the subtraction CT image provided in this application. Figure 4 As shown, the subtracted CT image is obtained by subtracting the plain scan CT image that has undergone spatial coordinate transformation from the enhanced CT image.

[0109] In this way, the automatic feature point extraction method based on machine learning does not require manual selection of feature points during image registration. It is fast and accurate, saving a lot of manpower and time, reducing the burden on medical staff, and avoiding inaccurate matching point selection caused by human factors. In addition, the target transformation matrix is ​​constructed using feature points in the heart area, which can keep the focus on the key areas of the heart, and more importantly, the registration of the coronary artery area. Compared with the transformation matrix construction based on global points, this solution can pay more attention to the heart area, making the final subtraction result more accurate.

[0110] An embodiment of the present application provides a method for subtracting a coronary CT image, the method comprising: acquiring an enhanced CT image and a plain scan CT image of a target patient; the enhanced CT image and the plain scan CT image are both CT images of the coronary arteries of the target patient; inputting the enhanced CT image into a pre-trained first feature point extraction model to extract a first feature point set of the enhanced CT image; inputting the plain scan CT image into a pre-trained second feature point extraction model to extract a second feature point set of the plain scan CT image; determining a target transformation matrix between the first feature point set and the second feature point set through iterative calculation based on the first feature point set and the second feature point set; performing coordinate transformation processing based on the plain scan CT image and the target transformation matrix to obtain a transformed plain scan CT image, so as to achieve image registration between the plain scan CT image and the enhanced CT image; and subtracting the grayscale values ​​of corresponding pixels of the transformed plain scan CT image from the grayscale values ​​of all pixels in the enhanced CT image to obtain a subtracted CT image.

[0111] Thus, this application extracts feature points from both the enhanced and plain CT images based on machine learning, pairs these feature points together to form point pairs, and constructs a target transformation matrix from these pairs. This transforms the plain CT image into the enhanced CT image space using the target transformation matrix, and finally subtracts the grayscale values ​​at corresponding locations in the enhanced and plain CT images, completing the closed-loop image subtraction task. Furthermore, this solution is fully automated, requiring no human intervention.

[0112] See also Figure 5 、 Figure 6 , Figure 5 This is one of the structural schematic diagrams of a coronary CT image subtraction device provided in an embodiment of the present application. Figure 6 This is a second structural diagram of a coronary CT image subtraction device provided in an embodiment of the present application. Figure 5 As shown in , the subtraction device 500 includes:

[0113] An acquisition module 510 is configured to acquire an enhanced CT image and a plain CT image of a target patient; both the enhanced CT image and the plain CT image are CT images of the coronary arteries of the target patient;

[0114] A first extraction module 520 is configured to input the enhanced CT image into a pre-trained first feature point extraction model to extract a first feature point set of the enhanced CT image;

[0115] A second extraction module 530 is configured to input the plain scan CT image into a pre-trained second feature point extraction model to extract a second feature point set of the plain scan CT image;

[0116] a matrix determination module 540 for determining a target transformation matrix between the first feature point set and the second feature point set through iterative calculation based on the first feature point set and the second feature point set;

[0117] a registration module 550 configured to perform coordinate transformation processing based on the plain scan CT image and the target transformation matrix to obtain a transformed plain scan CT image, so as to achieve image registration between the plain scan CT image and the enhanced CT image;

[0118] The subtraction module 560 is configured to subtract the grayscale values ​​of corresponding pixels in the transformed plain scan CT image from the grayscale values ​​of all pixels in the enhanced CT image to obtain a subtracted CT image.

[0119] Optional, such as Figure 6 As shown, the subtraction device 500 further includes a model building module 570, and the model building module 570 is used to:

[0120] Acquire a CT image training sample set; the CT image training sample set includes enhanced CT images and plain CT images of coronary arteries of multiple patients to be trained, and each CT image in the CT image training sample set has a corresponding feature point set label;

[0121] Determining the enhanced CT image to be trained as a first input feature of a first neural network, determining the feature point set label corresponding to the enhanced CT image to be trained as a first output feature of the first neural network, determining the plain scan CT image to be trained as a second input feature of a second neural network, and determining the feature point set label corresponding to the plain scan CT image to be trained as a second output feature of the second neural network;

[0122] Based on the first input feature, the first output feature, the second input feature, and the second output feature, performing alternating iterative training on the first neural network and the second neural network using a back gradient propagation algorithm; wherein, during the alternating iterative training, network parameters of the first neural network and the second neural network are shared;

[0123] When the first neural network and the second neural network reach a convergence state, the training is stopped to obtain a first feature point extraction model and a second feature point extraction model.

[0124] Optionally, when the matrix determination module 540 is used to determine the target transformation matrix between the first feature point set and the second feature point set through iterative calculation based on the first feature point set and the second feature point set, the matrix determination module 540 is used to:

[0125] Obtaining an initial transformation matrix; the initial transformation matrix includes multiple parameters corresponding to multiple transformation types, each parameter being set to a random initial value;

[0126] performing an initial transformation on the feature points in the second feature point set based on the initial transformation matrix to obtain a transformed second feature point set;

[0127] Setting an objective function based on the coordinates of the second feature point in the second feature point set after the transformation and the coordinates of the first feature point in the first feature point set; the objective function is used to determine the sum of the Euclidean distances between the second feature point and the first feature point after the transformation;

[0128] Based on the objective function, the random initial value corresponding to each parameter in the initial transformation matrix is ​​iteratively updated until the value of the objective function is less than a preset threshold, and the updating is stopped to obtain the target transformation matrix.

[0129] Optionally, the first neural network and the second neural network have the same initial network parameters.

[0130] Optionally, when the model construction module 570 is used to perform alternating iterative training on the first neural network and the second neural network using a back gradient propagation algorithm based on the first input feature, the first output feature, the second input feature, and the second output feature, the model construction module 570 is used to:

[0131] In each iteration, a first loss function is determined based on the first input feature and the first output feature, and a second loss function is determined based on the second input feature and the second output feature;

[0132] Alternately selecting the first loss function or the second loss function as a target loss function, and determining a target neural network corresponding to the selected target loss function; wherein the target neural network corresponding to the first loss function is a first neural network, and the target neural network corresponding to the second loss function is a second neural network;

[0133] Based on the target loss function, the network parameters of the target neural network are updated using the back gradient propagation algorithm, and the network parameters of the other neural network are updated at the same time to make the network parameters of the two neural networks consistent.

[0134] Optionally, the transformation type includes at least translation transformation, rotation transformation, perspective transformation and scaling transformation.

[0135] See also Figure 7 , Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 7 As shown in FIG, the electronic device 700 includes a processor 710 , a memory 720 and a bus 730 .

[0136] The memory 720 stores machine-readable instructions executable by the processor 710. When the electronic device 700 is running, the processor 710 communicates with the memory 720 via the bus 730. When the machine-readable instructions are executed by the processor 710, the above-mentioned Figures 1 to 4 The specific implementation of the steps in the method embodiment shown can be found in the method embodiment and will not be repeated here.

[0137] The embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the computer program can execute the above-mentioned Figures 1 to 4 The specific implementation of the steps in the method embodiment shown can be found in the method embodiment and will not be repeated here.

[0138] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0139] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interface, the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.

[0140] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0141] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0142] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0143] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The scope of protection of the present application is not limited thereto. Although the present application has been described in detail with reference to the above-mentioned embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-mentioned embodiments within the technical scope disclosed in the present application, or perform equivalent replacements for some of the technical features thereof. These modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A method for subtracting coronary CT images, characterized in that: The subtraction method comprises: Acquire an enhanced CT image and a plain CT image of a target patient; wherein the enhanced CT image and the plain CT image are both CT images of the coronary arteries of the target patient; Inputting the enhanced CT image into a pre-trained first feature point extraction model to extract a first feature point set of the enhanced CT image; and determining a predicted label and spatial coordinates of each first feature point; the first feature point extraction model is determined by training a first neural network; Inputting the plain scan CT image into a pre-trained second feature point extraction model to extract a second feature point set of the plain scan CT image; and determining a predicted label and spatial coordinates of each second feature point; the number of feature points in the second feature point set is the same as the number of feature points in the first feature point set; the second feature point extraction model is determined by training a second neural network; the first neural network and the second neural network are two sub-networks in a twin neural network; the first neural network and the second neural network are alternately iteratively trained, and during the alternately iterative training, the network parameters of the first neural network and the second neural network are shared; the initial network parameters of the first neural network and the second neural network are the same; Determining a target transformation matrix between the first feature point set and the second feature point set through iterative calculation based on the first feature point set and the second feature point set; performing coordinate transformation processing based on the plain scan CT image and the target transformation matrix to obtain a transformed plain scan CT image, so as to achieve image registration between the plain scan CT image and the enhanced CT image; The grayscale values ​​of all pixels in the enhanced CT image are subtracted from the grayscale values ​​of corresponding pixels in the transformed plain scan CT image to obtain a subtraction CT image.

2. The subtraction method according to claim 1, wherein: The first feature point extraction model and the second feature point extraction model are constructed by the following steps: Acquire a CT image training sample set; the CT image training sample set includes enhanced CT images and plain CT images of coronary arteries of multiple patients to be trained, and each CT image in the CT image training sample set has a corresponding feature point set label; Determining the enhanced CT image to be trained as a first input feature of a first neural network, determining the feature point set label corresponding to the enhanced CT image to be trained as a first output feature of the first neural network, determining the plain scan CT image to be trained as a second input feature of a second neural network, and determining the feature point set label corresponding to the plain scan CT image to be trained as a second output feature of the second neural network; Based on the first input feature, the first output feature, the second input feature, and the second output feature, performing alternating iterative training on the first neural network and the second neural network using a back gradient propagation algorithm; When the first neural network and the second neural network reach a convergence state, the training is stopped to obtain a first feature point extraction model and a second feature point extraction model.

3. The subtraction method according to claim 1, wherein: The determining, based on the first feature point set and the second feature point set, a target transformation matrix between the first feature point set and the second feature point set through iterative calculation includes: Obtaining an initial transformation matrix; the initial transformation matrix includes multiple parameters corresponding to multiple transformation types, each parameter being set to a random initial value; performing an initial transformation on the feature points in the second feature point set based on the initial transformation matrix to obtain a transformed second feature point set; Setting an objective function based on the coordinates of the second feature point in the second feature point set after the transformation and the coordinates of the first feature point in the first feature point set; the objective function is used to determine the sum of the Euclidean distances between the second feature point and the first feature point after the transformation; Based on the objective function, the random initial value corresponding to each parameter in the initial transformation matrix is ​​iteratively updated until the value of the objective function is less than a preset threshold, and the updating is stopped to obtain the target transformation matrix.

4. The subtraction method according to claim 2, wherein: The alternately iterative training of the first neural network and the second neural network using a back gradient propagation algorithm based on the first input feature, the first output feature, the second input feature, and the second output feature includes: In each iteration, a first loss function is determined based on the first input feature and the first output feature, and a second loss function is determined based on the second input feature and the second output feature; Alternately selecting the first loss function or the second loss function as a target loss function, and determining a target neural network corresponding to the selected target loss function; wherein the target neural network corresponding to the first loss function is a first neural network, and the target neural network corresponding to the second loss function is a second neural network; Based on the target loss function, the network parameters of the target neural network are updated using the back gradient propagation algorithm, and the network parameters of the other neural network are updated at the same time to make the network parameters of the two neural networks consistent.

5. The subtraction method according to claim 3, wherein: The transformation types include at least translation transformation, rotation transformation, perspective transformation and scaling transformation.

6. A coronary CT image subtraction device, characterized in that: The subtraction device comprises: An acquisition module, configured to acquire an enhanced CT image and a plain CT image of a target patient; the enhanced CT image and the plain CT image are both CT images of the coronary arteries of the target patient; a first extraction module, configured to input the enhanced CT image into a pre-trained first feature point extraction model to extract a first feature point set of the enhanced CT image; and determine a predicted label and spatial coordinates of each first feature point; the first feature point extraction model is determined by training a first neural network; a second extraction module, configured to input the plain scan CT image into a pre-trained second feature point extraction model, extract a second feature point set of the plain scan CT image, and determine a predicted label and spatial coordinates of each second feature point; the number of feature points in the second feature point set is the same as the number of feature points in the first feature point set; the second feature point extraction model is determined by training a second neural network; the first neural network and the second neural network are two subnetworks in a twin neural network; the first neural network and the second neural network are trained alternately and iteratively, and during the alternate iterative training, the network parameters of the first neural network and the second neural network are shared; and the initial network parameters of the first neural network and the second neural network are the same; a matrix determination module, configured to determine, based on the first feature point set and the second feature point set, a target transformation matrix between the first feature point set and the second feature point set through iterative calculation; a registration module, configured to perform coordinate transformation processing based on the plain scan CT image and the target transformation matrix to obtain a transformed plain scan CT image, so as to achieve image registration between the plain scan CT image and the enhanced CT image; The subtraction module is configured to subtract the grayscale values ​​of corresponding pixels in the transformed plain scan CT image from the grayscale values ​​of all pixels in the enhanced CT image to obtain a subtracted CT image.

7. The subtraction device according to claim 6, characterized in that The subtraction device further includes a model building module, and the model building module is used to: Acquire a CT image training sample set; the CT image training sample set includes enhanced CT images and plain CT images of coronary arteries of multiple patients to be trained, and each CT image in the CT image training sample set has a corresponding feature point set label; Determining the enhanced CT image to be trained as a first input feature of a first neural network, determining the feature point set label corresponding to the enhanced CT image to be trained as a first output feature of the first neural network, determining the plain scan CT image to be trained as a second input feature of a second neural network, and determining the feature point set label corresponding to the plain scan CT image to be trained as a second output feature of the second neural network; Based on the first input feature, the first output feature, the second input feature, and the second output feature, performing alternating iterative training on the first neural network and the second neural network using a back gradient propagation algorithm; When the first neural network and the second neural network reach a convergence state, the training is stopped to obtain a first feature point extraction model and a second feature point extraction model.

8. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and the machine-readable instructions are executed by the processor to execute the steps of the subtraction method as described in any one of claims 1 to 5.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the subtraction method according to any one of claims 1 to 5 are executed.

Images