Respiration compensation method, device and equipment and storage medium

Abstract

The invention discloses a respiration compensation method, device and equipment and a storage medium, and the method decomposes respiration movement into rigid displacement movement and non-rigid deformation movement, and performs compensation and correction according to the characteristics of the two kinds of movement. For rigid motion, the blood vessel is wrapped by liver parenchyma, the rigid motion of the blood vessel can be replaced by the liver, the mass center track of the liver area in the X image sequence is tracked, a motion track curve of the blood vessel can be rapidly obtained, and an initial blood vessel path map is driven. Next, for the deformation correction of the blood vessel, calculating the bending energy of a guide wire in the lumen of the blood vessel, and screening out the structural characteristics of the part with higher energy, so as to drive the elastic registration of an image region at different moments, and complete the non-rigid deformation compensation of the blood vessel. The method can be used for effectively, quickly and continuously tracking the surgical instruments in the blood vessel and reducing the dependence on a contrast agent.

Description

Technical Field

[0001] The present invention relates to the field of medical technology, and in particular to a breathing compensation method, device, equipment and storage medium based on adaptive tracking and bending energy modeling. Background Art

[0002] X-ray image guidance is widely used in liver vascular interventional surgery. Specifically, X-ray image-guided liver vascular interventional surgery requires angiography to determine the position and posture of the hepatic vessels in order to continuously track the surgical instruments in the blood vessels. Unlike other vascular interventional surgeries (such as coronary artery, aortic, and hepatic artery interventions), the contrast agent in the hepatic vessels is quickly lost during surgery, and the hepatic vascular branches cannot be continuously seen. In addition, intraoperative breathing can cause these branches to slide and deform unpredictably, making it extremely challenging to predict the posture of the branches and the position of the positioning instruments in the blood vessels, increasing the risks and time consumption associated with the intervention.

[0003] To address these challenges, vascular prediction methods are often used to generate a roadmap for interventional guidance, eliminating the need for contrast agents. Currently, some studies have used 3D / 2D registration techniques to project segmented vessels in preoperative computed tomography (CT) volumes onto X-ray image sequences for prediction. However, the time-consuming nature of 3D / 2D registration between CT images and each frame of the X-ray image sequence makes it impossible to meet the rapid implementation of interventional guidance.

[0004] Many studies have explored respiratory compensation, classifying the approaches as either external devices or image sequence-based respiratory motion modeling. External devices include respiratory belts, surface markers, and electromagnetic sensors. While estimating respiratory motion using these devices is simple and applicable, it affects clinical settings. In contrast, image-based approaches estimate respiratory motion by extracting structures from image sequences.

[0005] As a representative of anatomical structures, the diaphragm is often used to analyze respiratory motion. For example, James et al. proposed an appearance-based tracking algorithm to compensate for the overall liver motion caused by breathing and used the diaphragm to achieve perspective enhancement of hepatic vascular embolism. Wagner et al. estimated the respiratory state based on the diaphragm position in non-contrast images and fitted a linear function between the vascular affine transformation and the respiratory state. King et al. developed a respiratory motion correction system for X-ray image-guided cardiac catheterization and updated the roadmap by tracking the diaphragm motion in X-ray fluoroscopic images. Fischer et al. achieved motion compensation by tracking the up and down sliding motion of the diaphragm for clinical use. They are sensitive to the integrity and imaging quality of the diaphragm in X-ray images. Therefore, despite their time efficiency, diaphragm-based methods still need to improve accuracy and robustness when the diaphragm is incomplete and imaging noise is disturbed.

[0006] Unlike diaphragm-based respiratory modeling, interventional device modeling focuses on learning and predicting the motion of vascular branches during surgery. For example, Ambrosini et al. used a hidden Markov model (HMM) to track the catheter tip, enabling the superposition of 3D blood vessels on each frame of an X-ray fluorescence image sequence during free breathing. Ma et al. introduced a novel dynamic coronary artery prediction method based on Bayesian filtering and catheter tip tracking. They used electrocardiogram (ECG) data obtained from percutaneous coronary intervention (PCI) surgery to establish the correspondence between the catheter tip in image frames under different motion states and dynamically predict the coronary artery roadmap in subsequent image frames. Vernikouskaya et al. used a convolutional neural network (CNN) to learn the displacement of the catheter tip and associate the vascular motion with the respiratory motion predicted by CNN through ECG data.

[0007] However, the above methods all have the disadvantage of being unstable in predicting the position of hepatic blood vessels during the patient's free breathing during surgery. Summary of the invention

[0008] In view of the above problems, the present invention provides a breathing compensation method, device, equipment and storage medium for overcoming the above problems or at least partially solving the above problems. The current position and morphology of the blood vessel can be accurately predicted in the presence of intraoperative breathing and interventional guidewire and without contrast agent.

[0009] The present invention provides the following scheme:

[0010] A breathing compensation method, comprising:

[0011] Acquire a sequence of continuous frame X-ray images of the surgical area during surgery;

[0012] Extracting and obtaining a plurality of continuous liver mask images from the X-ray image sequence;

[0013] Calculating the position of the liver centroid in each of the liver mask images respectively;

[0014] Calculating the relative displacement movement between frames using the position of the liver mass center corresponding to each of the liver mask images, so as to calculate and fit a respiratory curve;

[0015] Using the respiratory curve in combination with the initial vascular roadmap, dynamically predicting the vascular posture in each frame of the X-ray image sequence to obtain a number of rigid motion compensated vascular roadmaps;

[0016] Refining the guidewire in the blood vessel lumen of each X-ray image in the X-ray image sequence into a plurality of trajectory points, and using Bezier curve estimation to implement bending energy modeling to obtain energy characteristics of each of the trajectory points;

[0017] Selecting a number of control points for guidewire bending energy sampling in each X-ray image according to the energy characteristics of each trajectory point;

[0018] Matching the control points corresponding to each of the X-ray images;

[0019] The matched control points are used to guide the rigid motion compensated blood vessel roadmaps to form a deformation field, so as to realize elastic registration of image regions at different times, complete non-rigid motion compensation of blood vessels and obtain continuous blood vessel prediction results.

[0020] Preferably: a U-Net-based segmentation network is used to extract a plurality of continuous liver mask images from the X-ray image sequence; and a loss function defined according to a Dice similarity coefficient is used during the training process of the U-Net-based segmentation network.

[0021] Preferably, the breathing curve is obtained by calculating and fitting the following formula:

[0022]

[0023] Where: Indicates the breathing movement along the head and feet direction, Indicates the breathing movement along the left and right direction, I m (x,y) represents the pixel grayscale of the liver mask image, x, y represent the row coordinates and column coordinates of the image, size represents the image scale, Indicates that x is the sum of the products of the row pixel coordinates and the corresponding pixel grayscale. It means that y is the sum of the products of the column pixel coordinates and the corresponding pixel grayscale.

[0024] Preferably: a dynamic time planning algorithm is used to match the control points at different times.

[0025] Preferably, using the matched control points to guide the rigid motion compensated blood vessel roadmaps to form a deformation field comprises:

[0026] Based on B-spline interpolation, a bending energy operator is introduced to perform deformation according to the maximum energy feature in the calculation area of ​​the interpolation control points supporting the deformation.

[0027] Preferably: given a fixed two-dimensional image F, the image internal coordinates are θ = (x1, y1), and the image intensity is f = F(θ); a moving image M, and the corresponding coordinates are φ = (x2, y2) and the image intensity is m = M(φ), the two images are registered by the following loss function:

[0028]

[0029] In the formula: Ψ(f,m) represents, λ represents weight, and S represents smoothness.

[0030] Preferably, the smoothness S is expressed by the following formula:

[0031]

[0032] Where: v represents the dense vector field defined for each pixel θ∈Ω.

[0033] A breathing compensation device, used to perform the above breathing compensation method, the device comprising:

[0034] An image sequence acquisition unit, used for acquiring a sequence of continuous frame X-ray images of the surgical area during surgery;

[0035] A liver mask image extraction unit, used for extracting a plurality of continuous liver mask images from the X-ray image sequence;

[0036] A liver centroid position calculation unit, used for respectively calculating the position of the liver centroid in each of the liver mask images;

[0037] A respiratory curve fitting unit, used to calculate the relative displacement movement between frames using the position of the liver mass center corresponding to each of the liver mask images, so as to calculate and fit the respiratory curve;

[0038] A rigid motion compensation unit, used to dynamically predict the blood vessel posture in each frame of the X-ray image sequence by using the respiratory curve in combination with an initial blood vessel roadmap to obtain a plurality of rigid motion compensated blood vessel roadmaps;

[0039] A guidewire trajectory point extraction unit, used to refine the guidewire in the blood vessel lumen of each X-ray image in the X-ray image sequence into a plurality of trajectory points, and use Bezier curve estimation to implement bending energy modeling to obtain energy characteristics of each of the trajectory points;

[0040] A control point screening unit, used for screening a number of control points for guide wire bending energy sampling in each of the X-ray images according to the energy characteristics of each of the trajectory points;

[0041] A matching unit, used for matching the control points corresponding to each of the X-ray images;

[0042] The non-rigid motion compensation unit is used to use the matched control points to guide the rigid motion compensated blood vessel roadmaps to form a deformation field, realize elastic registration of image areas at different times, complete the non-rigid motion compensation of blood vessels, and obtain continuous blood vessel prediction results.

[0043] A breathing compensation device, the device comprising a processor and a memory:

[0044] The memory is used to store program codes and transmit the program codes to the processor;

[0045] The processor is used to execute the above-mentioned breathing compensation method according to the instructions in the program code.

[0046] A computer-readable storage medium is used to store program codes, and the program codes are used to execute the above-mentioned breathing compensation method.

[0047] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0048] The embodiments of the present application provide a respiratory compensation method, device, equipment and storage medium, which decomposes respiratory motion into rigid displacement motion and non-rigid deformation motion, and compensates and corrects the characteristics of these two motions respectively. For rigid motion, the blood vessels are wrapped by the liver parenchyma, and the rigid motion of the blood vessels can be replaced by the liver. By tracking the centroid trajectory of the liver area in the X-ray image sequence, the motion trajectory curve of the blood vessels can be quickly obtained, and the initial blood vessel roadmap can be driven. Next, for the deformation correction of the blood vessels, the bending energy of the guide wire in the lumen of the blood vessel is calculated, and the structural features of the higher energy part are screened out to drive the elastic alignment of the image area at different times, and complete the non-rigid deformation compensation of the blood vessels. This method can effectively and quickly continuously track surgical instruments in blood vessels and reduce dependence on contrast agents.

[0049] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0051] Figure 1 is a flow chart of a breathing compensation method provided by an embodiment of the present invention;

[0052] Figure 2 It is a framework diagram of a breathing compensation method implementation process provided by an embodiment of the present invention;

[0053] Figure 3 is a schematic diagram of a breathing compensation device provided by an embodiment of the present invention;

[0054] Figure 4 It is a schematic diagram of a breathing compensation device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0055] The technical scheme in the embodiment of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all of the embodiments. Based on the embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0056] See also Figure 1 , which is provided by an embodiment of the present invention, such as Figure 1 As shown, the method may include:

[0057] S101: Acquire a sequence of continuous frame X-ray images of the surgical area during surgery;

[0058] S102: extracting a plurality of continuous liver mask images from the X-ray image sequence; in specific implementation, the embodiment of the present application may provide a method of extracting a plurality of continuous liver mask images from the X-ray image sequence using a U-Net-based segmentation network; and a loss function defined according to a Dice similarity coefficient is used during training of the U-Net-based segmentation network.

[0059] S103: calculating the position of the liver centroid in each of the liver mask images respectively;

[0060] S104: Calculate the relative displacement movement between frames using the position of the liver mass center corresponding to each of the liver mask images, so as to calculate and fit the respiratory curve; in specific implementation, the embodiment of the present application can provide a respiratory curve calculated and fitted by the following formula:

[0061]

[0062] Where: Indicates the breathing movement along the head and feet direction, Indicates the breathing movement along the left and right direction, I m (x,y) represents the pixel grayscale of the liver mask image, x, y represent the row coordinates and column coordinates of the image, size represents the image scale, Indicates that x is the sum of the products of the row pixel coordinates and the corresponding pixel grayscale. It means that y is the sum of the products of the column pixel coordinates and the corresponding pixel grayscale.

[0063] S105: dynamically predicting the blood vessel posture in each frame of the X-ray image sequence by using the respiratory curve in combination with the initial blood vessel roadmap to obtain a plurality of rigid motion compensated blood vessel roadmaps;

[0064] S106: Refining the guidewire in the blood vessel lumen of each X-ray image in the X-ray image sequence into a plurality of trajectory points, and using Bezier curve estimation to implement bending energy modeling to obtain energy characteristics of each trajectory point;

[0065] S107: screening out a number of control points for guide wire bending energy sampling in each of the X-ray images according to the energy characteristics of each of the trajectory points;

[0066] S108: Match the control points corresponding to each of the X-ray images. In specific implementation, the embodiment of the present application may provide an algorithm using dynamic time planning to match the control points at different times.

[0067] S109: using the matched control points to guide the rigid motion compensated blood vessel roadmaps to form a deformation field, to achieve elastic registration of image regions at different times, to complete non-rigid motion compensation of blood vessels, and to obtain continuous blood vessel prediction results.

[0068] In specific implementation, the embodiments of the present application may provide for introducing a bending energy operator based on B-spline interpolation, and performing deformation according to the maximum energy feature in the calculation area of ​​the interpolation control points supporting the deformation.

[0069] Furthermore, given a fixed 2D image F, with image internal coordinates θ = (x1, y1) and image intensity f = F(θ); a moving image M, with corresponding coordinates φ = (x2, y2) and image intensity m = M(φ), the two images are registered using the following loss function:

[0070]

[0071] In the formula: Ψ(f,m) represents, λ represents weight, and S represents smoothness.

[0072] The smoothness S is expressed by the following formula:

[0073]

[0074] Where: v represents the dense vector field defined for each pixel θ∈Ω.

[0075] The respiratory compensation method provided in the embodiment of the present application utilizes adaptive tracking and bending energy modeling to achieve stable vascular prediction under free breathing. First, a rigid displacement compensation method between image frames based on image domain conversion and adaptive centroid tracking is introduced. It fits the respiratory curve from the input X-ray image sequence to provide a temporal motion prior for aligning the vascular roadmap between image frames. Secondly, a compensation method based on bending energy modeling is proposed to correct respiratory deformation, using the energy characteristics of the guidewire to drive non-rigid registration. The control points sampled according to the bending energy guide the local image to form a deformation field, which helps to dynamically superimpose the deformed vascular roadmap on the X-ray image sequence. The effective and rapid compensation that can be achieved by this method has the potential to improve liver intervention and reduce dependence on contrast agents.

[0076] The method provided in the embodiments of the present application is described in detail below.

[0077] In order to complete the respiratory compensation on the X-ray image sequence and realize the continuous prediction of the hepatic vessels, such as Figure 2 As shown, the method framework diagram constructed by the embodiment of the present application mainly includes two parts. The framework extracts the liver area from the input X-ray image and adaptively tracks the center of mass position in multiple frames. By calculating the relative displacement of the center of mass between frames, the respiratory motion curve is calculated and fitted, thereby realizing dynamic vascular position prediction.

[0078] Then, it performs non-rigid deformation compensation by modeling the bending energy of the guidewire. After calculating the bending energy of the guidewire structure, energy feature control points are selected to guide non-rigid registration, and a deformation field is generated around the guidewire according to the local image intensity change, finally achieving continuous prediction of vascular morphology.

[0079] 1. Rigid motion compensation based on adaptive center of mass tracking.

[0080] The rapid calculation of rigid displacement between image frames relies on the tracking of the image centroid, but the accuracy of this process is easily affected by the position of the diaphragm and the quality of X-ray imaging. In order to dynamically track the centroid in X-ray image-guided liver intervention, the liver region is first extracted. We obtain the liver mask from the X-ray image sequence through the network segmentation method. In order to obtain continuous liver segmentation results from the input X-ray image sequence, we train a U-Net-based segmentation network. During the training process, we obtain the gold standard liver segmentation results through doctors, and at the same time improve the intensity difference between the liver region and the background region through image preprocessing. Our loss function is defined based on the Dice similarity coefficient:

[0081]

[0082] Among them B P and BGT They represent the predicted results and segmentation gold standard of the liver, respectively, and they have the same scale as the input X-ray image.

[0083] After obtaining continuous liver mask images I m (x, y), where m represents the respiratory motion within a respiratory cycle. Each mask image is a binary image defined on a regular two-dimensional grid represented by an xy coordinate system. On this basis, we extract the position of the liver centroid to calculate the relative displacement between frames, and use their position trajectories to fit the respiratory curve. The formula is as follows:

[0084]

[0085]

[0086] Where: Indicates the breathing movement along the head and feet direction, Indicates the breathing movement along the left and right direction, I m (x,y) represents the pixel grayscale of the liver mask image, x, y represent the row coordinates and column coordinates of the image, size represents the image scale, Indicates that x is the sum of the products of the row pixel coordinates and the corresponding pixel grayscale. It means that y is the sum of the products of the column pixel coordinates and the corresponding pixel grayscale.

[0087] In order to achieve continuous respiratory motion compensation, it is necessary to calculate the relative motion between two adjacent frames within a respiratory cycle. The formula is as follows:

[0088]

[0089] in and Represents two adjacent moments.

[0090] In this process, displacement compensation is equivalent to applying an affine transformation to drive the vascular roadmap to move. Each element in the affine transformation matrix can be used and Finally, this rigid transformation matrix will be applied as the initial roadmap and the dynamic blood vessel prediction results will be obtained in subsequent image frames.

[0091] In clinical practice, an initial vascular roadmap is established on the first frame through 3D / 2D registration. Respiration compensation is then performed to adjust the position and posture of the vascular roadmap in subsequent frames. The essence of compensation is to guide the movement and deformation of the initial roadmap by modeling intraoperative breathing, which includes the rigid displacement movement of the blood vessels and the non-rigid vascular deformation.

[0092] 2. Non-rigid motion compensation based on bending energy modeling.

[0093] After rigid displacement compensation, dynamic vascular prediction is achieved. However, the nonparametric deformation between liver parenchyma and intrahepatic vessels during breathing still affects the prediction accuracy of vascular roadmap. To improve accuracy and computational efficiency, we use liver region images from guidewires and continuous X-ray images to achieve nonrigid deformation field estimation.

[0094] As the guidewire advances along the lumen of a vascular branch, it stores mechanical energy in the material by bending. The bending elasticity of the structure brings about the local maximum entropy (LME), which makes the curved structure more easily deformable during breathing, and the residual area will translate with the breathing trend.

[0095] Therefore, the present embodiment refines the guidewire into trajectory points and uses the curvature of each point to characterize the bending energy. The bending energy modeling is realized by using Bezier curve estimation to obtain the energy characteristics of each point on the guidewire, and the control point P is obtained by screening. i,j , then the matching between the control points at different times needs to be established. For each set of trajectory points that need to be matched, the dynamic time planning (DTW) algorithm is used to achieve matching. The corresponding point sets are represented as s i ={p1,p2,…,p m} and s j ={q1,q2,…,q n}.

[0096] The goal of DTW is to find a matching path in the matrix grid and point p in the two sequences i and dot q j When the continuity and monotonicity constraints are met, multiple paths can be obtained, where the formula for minimizing the regularization cost is:

[0097]

[0098] By recursively calculating the path D(i,j), continuous matching of two point set sequences is obtained.

[0099] Next, these matched feature control points are used to guide non-rigid registration. Here, a bending energy operator can be introduced based on B-spline interpolation, which performs deformation based on the maximum energy feature in the region calculated by the interpolation control points that support deformation. Given a fixed two-dimensional image F, the image internal coordinates are θ = (x1, y1), and the image intensity is f = F(θ). Then the moving image M also corresponds to the coordinates φ = (x2, y2) and the image intensity m = M(φ). The two images can be registered using the following loss function:

[0100] C=∑ T(θ)∈ΩΨ(f,m)+λS, (6)

[0101] The above formula can be optimized by evaluating the coordinate mapping M(θ) = θ + v through the similarity metric Ψ, where v is a dense vector field defined for each pixel θ∈Ω, assuming that it can achieve a one-to-one mapping from F to M. The smoothness S of v is added to C with a weight λ to drive M to a meaningful coordinate map.

[0102] Here the vector field v is composed of control points P i,j =p x ,p y Parameterization is achieved, where p x =[(p x ,0,0),…,(p x ,I,J)] and p y =[(p y ,0,0),…,(p y ,I,J)] is the definition for n = I × J control points, and the pixel spacing is r = r x ,r y .

[0103] The smoothness S can be expressed by the following formula:

[0104]

[0105] Among them, S is used to smooth the gradient change during the deformation process, for the control point P i,j The neighborhood within the local control range is smoothly deformed, and the deformed continuous blood vessel prediction result is obtained in the subsequent X-ray image.

[0106] In summary, the respiratory compensation method provided by the present application decomposes respiratory motion into rigid displacement motion and non-rigid deformation motion, and compensates and corrects the characteristics of these two motions respectively. For rigid motion, the blood vessels are wrapped by the liver parenchyma, and the rigid motion of the blood vessels can be replaced by the liver. By tracking the centroid trajectory of the liver area in the X-ray image sequence, the motion trajectory curve of the blood vessels can be quickly obtained, and the initial vascular roadmap can be driven. Next, for the deformation correction of the blood vessels, the bending energy of the guide wire in the vascular lumen is calculated, and the structural features of the higher energy part are screened out to drive the elastic alignment of the image area at different times, and complete the non-rigid deformation compensation of the blood vessels. This method can effectively and quickly continuously track surgical instruments in blood vessels and reduce dependence on contrast agents.

[0107] See also Figure 3 , the embodiment of the present application may also provide a breathing compensation device, such as Figure 3 As shown, the device may include:

[0108] An image sequence acquisition unit 301 is used to acquire a sequence of continuous frame X-ray images of the surgical area during surgery;

[0109] A liver mask image extraction unit 302, configured to extract a plurality of continuous liver mask images from the X-ray image sequence;

[0110] A liver centroid position calculation unit 303, used to calculate the position of the liver centroid in each of the liver mask images;

[0111] A respiratory curve fitting unit 304 is used to calculate the relative displacement movement between frames using the position of the liver mass center corresponding to each of the liver mask images, so as to calculate and fit the respiratory curve;

[0112] A rigid motion compensation unit 305 is used to dynamically predict the blood vessel posture in each frame of the X-ray image sequence by using the respiratory curve in combination with the initial blood vessel roadmap to obtain a number of rigid motion compensated blood vessel roadmaps;

[0113] A guidewire trajectory point extraction unit 306 is used to refine the guidewire in the blood vessel lumen of each X-ray image in the X-ray image sequence into a plurality of trajectory points, and use Bezier curve estimation to implement bending energy modeling to obtain energy characteristics of each of the trajectory points;

[0114] A control point screening unit 307, configured to screen a number of control points for sampling the guide wire bending energy in each of the X-ray images according to the energy characteristics of each of the trajectory points;

[0115] A matching unit 308, configured to match the control points corresponding to each of the X-ray images;

[0116] The non-rigid motion compensation unit 309 is used to use the matched control points to guide the rigid motion compensated blood vessel roadmaps to form a deformation field, realize elastic registration of image areas at different times, complete the non-rigid motion compensation of blood vessels, and obtain continuous blood vessel prediction results.

[0117] The embodiment of the present application may also provide a breathing compensation device, the device comprising a processor and a memory:

[0118] The memory is used to store program code and transmit the program code to the processor;

[0119] The processor is used to execute the steps of the above-mentioned breathing compensation method according to the instructions in the program code.

[0120] like Figure 4As shown, a breathing compensation device provided by an embodiment of the present application may include: a processor 10, a memory 11, a communication interface 12 and a communication bus 13. The processor 10, the memory 11 and the communication interface 12 communicate with each other through the communication bus 13.

[0121] In the embodiment of the present application, the processor 10 may be a central processing unit (CPU), an application specific integrated circuit, a digital signal processor, a field programmable gate array or other programmable logic devices.

[0122] The processor 10 may call a program stored in the memory 11. Specifically, the processor 10 may execute operations in an embodiment of the breathing compensation method.

[0123] The memory 11 is used to store one or more programs, which may include program codes, and the program codes include computer operation instructions. In the embodiment of the present application, the memory 11 at least stores programs for implementing the following functions:

[0124] Acquire a sequence of continuous frame X-ray images of the surgical area during surgery;

[0125] Extracting and obtaining a plurality of continuous liver mask images from the X-ray image sequence;

[0126] Calculating the position of the liver centroid in each of the liver mask images respectively;

[0127] Calculating the relative displacement movement between frames using the position of the liver mass center corresponding to each of the liver mask images, so as to calculate and fit a respiratory curve;

[0128] Using the respiratory curve in combination with the initial vascular roadmap, dynamically predicting the vascular posture in each frame of the X-ray image sequence to obtain a number of rigid motion compensated vascular roadmaps;

[0129] Refining the guidewire in the blood vessel lumen of each X-ray image in the X-ray image sequence into a plurality of trajectory points, and using Bezier curve estimation to implement bending energy modeling to obtain energy characteristics of each of the trajectory points;

[0130] Selecting a number of control points for guidewire bending energy sampling in each X-ray image according to the energy characteristics of each trajectory point;

[0131] Matching the control points corresponding to each of the X-ray images;

[0132] The matched control points are used to guide the rigid motion compensated blood vessel roadmaps to form a deformation field, so as to realize elastic registration of image regions at different times, complete the non-rigid motion compensation of blood vessels and obtain continuous blood vessel prediction results.

[0133] In one possible implementation, the memory 11 may include a program storage area and a data storage area, wherein the program storage area can store an operating system and application programs required for at least one function (such as a file creation function, a data reading and writing function), etc.; the data storage area can store data created during use, such as initialization data, etc.

[0134] In addition, the memory 11 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device.

[0135] The communication interface 12 may be an interface of a communication module, and is used to connect to other devices or systems.

[0136] Of course, it should be noted that Figure 4 The structure shown does not constitute a limitation on the breathing compensation device in the embodiment of the present application. In actual applications, the breathing compensation device may include Figure 4 More or fewer components than shown, or combinations of certain components.

[0137] The embodiment of the present application may also provide a computer-readable storage medium, wherein the computer-readable storage medium is used to store program codes, and the program codes are used to execute the steps of the above-mentioned breathing compensation method.

[0138] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0139] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0140] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.

[0141] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A breathing compensation method, characterized in that: include: Acquire a sequence of continuous frame X-ray images of the surgical area during surgery; Extracting and obtaining a plurality of continuous liver mask images from the X-ray image sequence; Calculating the position of the liver centroid in each of the liver mask images respectively; Calculating the relative displacement movement between frames using the position of the liver mass center corresponding to each of the liver mask images, so as to calculate and fit a respiratory curve; Using the respiratory curve in combination with the initial vascular roadmap, dynamically predicting the vascular posture in each frame of the X-ray image sequence to obtain a number of rigid motion compensated vascular roadmaps; Refining the guidewire in the blood vessel lumen of each X-ray image in the X-ray image sequence into a plurality of trajectory points, and using Bezier curve estimation to implement bending energy modeling to obtain energy characteristics of each of the trajectory points; Selecting a number of control points for guidewire bending energy sampling in each X-ray image according to the energy characteristics of each trajectory point; Matching the control points corresponding to each of the X-ray images; The matched control points are used to guide the rigid motion compensated blood vessel roadmaps to form a deformation field, so as to realize elastic registration of image regions at different times, complete non-rigid motion compensation of blood vessels and obtain continuous blood vessel prediction results.

2. The breathing compensation method according to claim 1, characterized in that: A U-Net-based segmentation network is used to extract a plurality of continuous liver mask images from the X-ray image sequence; a loss function defined according to the Dice similarity coefficient is used during the training process of the U-Net-based segmentation network.

3. The breathing compensation method according to claim 1, characterized in that: The breathing curve is obtained by calculation and fitting using the following formula: Where: Indicates the breathing movement along the head and feet direction, Indicates the breathing movement along the left and right direction, I m (x,y) represents the pixel grayscale of the liver mask image, x, y represent the row coordinates and column coordinates of the image, size represents the image scale, Indicates that x is the sum of the products of the row pixel coordinates and the corresponding pixel grayscale. It means that y is the sum of the products of the column pixel coordinates and the corresponding pixel grayscale.

4. The breathing compensation method according to claim 1, characterized in that: A dynamic time planning algorithm is used to match the control points at different times.

5. The breathing compensation method according to claim 1, characterized in that: Using the matched control points to guide the rigid motion compensated blood vessel roadmaps to form a deformation field includes: Based on B-spline interpolation, a bending energy operator is introduced to perform deformation according to the maximum energy feature in the calculation area of ​​the interpolation control points supporting the deformation.

6. The breathing compensation method according to claim 5, characterized in that: Given a fixed 2D image F, with image internal coordinates θ = (x1, y1) and image intensity f = F(θ); a moving image M, with corresponding coordinates φ = (x2, y2) and image intensity m = M(φ), the two images are aligned using the following loss function: Where: Ψ(f,m) represents, λ represents weight, and S represents smoothness.

7. The breathing compensation method according to claim 5, characterized in that: The smoothness S is expressed by the following formula: Where: v represents the dense vector field defined for each pixel θ∈Ω.

8. A breathing compensation device, characterized in that: For executing the breathing compensation method according to any one of claims 1 to 7, the device comprises: An image sequence acquisition unit, used for acquiring a sequence of continuous frame X-ray images of the surgical area during surgery; A liver mask image extraction unit, used for extracting a plurality of continuous liver mask images from the X-ray image sequence; A liver centroid position calculation unit, used for respectively calculating the position of the liver centroid in each of the liver mask images; A respiratory curve fitting unit, used to calculate the relative displacement movement between frames using the position of the liver mass center corresponding to each of the liver mask images, so as to calculate and fit the respiratory curve; A rigid motion compensation unit, used to dynamically predict the blood vessel posture in each frame of the X-ray image sequence by using the respiratory curve in combination with an initial blood vessel roadmap to obtain a plurality of rigid motion compensated blood vessel roadmaps; A guidewire trajectory point extraction unit, used to refine the guidewire in the blood vessel lumen of each X-ray image in the X-ray image sequence into a plurality of trajectory points, and use Bezier curve estimation to implement bending energy modeling to obtain energy characteristics of each of the trajectory points; A control point screening unit, used for screening a number of control points for guide wire bending energy sampling in each of the X-ray images according to the energy characteristics of each of the trajectory points; A matching unit, used for matching the control points corresponding to each of the X-ray images; The non-rigid motion compensation unit is used to use the matched control points to guide the rigid motion compensated blood vessel roadmaps to form a deformation field, realize elastic registration of image areas at different times, complete the non-rigid motion compensation of blood vessels, and obtain continuous blood vessel prediction results.

9. A breathing compensation device, characterized in that: The device comprises a processor and a memory: The memory is used to store program codes and transmit the program codes to the processor; The processor is used to execute the breathing compensation method according to any one of claims 1-7 according to the instructions in the program code.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store program codes, and the program codes are used to execute the breathing compensation method according to any one of claims 1 to 7.

Images