Breathing compensation method, apparatus, device, and storage medium

Abstract

The application discloses a kind of respiratory compensation method, device, equipment and storage medium, the method is decomposed into rigid displacement movement and non-rigid deformation movement for respiratory movement, and compensation and correction are carried out respectively for the characteristics of the two movements.For rigid movement, blood vessels are wrapped by liver parenchyma, and the rigid movement of blood vessels can be replaced by the liver. By tracking the centroid trajectory of the liver region in the X image sequence, the motion trajectory curve of the blood vessel can be quickly obtained, and the initial blood vessel road map is driven. Next, for the deformation correction of the blood vessel, the bending energy of the guide wire in the blood vessel lumen is calculated, and the structural features with higher energy are selected to drive the elastic registration of the image region at different times, and the non-rigid deformation compensation of the blood vessel is completed. The method can effectively and quickly track the surgical instrument in the blood vessel and reduce the dependence on contrast medium.

Description

Technical Field

[0001] The present invention relates to the field of medical technology, and in particular to a breathing compensation method, device, equipment and storage medium based on adaptive tracking and bending energy modeling. Background Art

[0002] X-ray image guidance is widely used in liver vascular interventional surgery. Specifically, X-ray image-guided liver vascular interventional surgery requires angiography to determine the position and posture of the liver blood vessels in order to continuously track the surgical instruments in the blood vessels. Unlike other vascular interventional surgeries (such as coronary artery, aortic and hepatic artery interventions), the contrast agent in the liver blood vessels is quickly lost during the operation, and the liver vascular branches cannot be continuously seen. In addition, intraoperative breathing can cause these branches to slide and deform unpredictably, making it extremely challenging to predict the posture of the branches and the position of the positioning instruments in the blood vessels, increasing the risks and time consumption associated with the intervention.

[0003] To address these challenges, vascular prediction methods are often used to generate roadmaps for interventional guidance, eliminating the need for contrast agents. Currently, some studies have used 3D / 2D registration techniques to project segmented vessels from preoperative computed tomography (CT) volumes onto X-ray image sequences for prediction. However, the time-consuming nature of 3D / 2D registration between CT images and each frame of the X-ray image sequence makes it unsuitable for rapid implementation of interventional guidance.

[0004] Numerous studies have explored respiratory compensation, with approaches categorized as either based on external devices or image sequences for respiratory motion modeling. External devices include respiratory belts, surface markers, and electromagnetic sensors. While estimating respiratory motion using these devices is simple and applicable, it can impact clinical practice. In contrast, image-based methods estimate respiratory motion by extracting structure from image sequences.

[0005] The diaphragm, as a representative anatomical structure, is often used to analyze respiratory motion. For example, James et al. proposed an appearance-based tracking algorithm to compensate for the overall liver motion caused by respiration and used the diaphragm to achieve fluoroscopic enhancement of hepatic vascular embolism. Wagner et al. estimated the respiratory state based on the diaphragm position in non-contrast images and fitted a linear function between the vascular affine transformation and the respiratory state. King et al. developed a respiratory motion correction system for X-ray image-guided cardiac catheterization and updated the roadmap by tracking the diaphragm motion in X-ray fluoroscopic images. Fischer et al. achieved motion compensation by tracking the up and down sliding motion of the diaphragm for clinical use. They are sensitive to the integrity and imaging quality of the diaphragm in X-ray images. Therefore, although they are time-efficient, diaphragm-based methods still need to improve accuracy and robustness when the diaphragm is incomplete and imaging noise is disturbed.

[0006] Unlike diaphragm-based respiratory modeling, interventional device modeling focuses on learning and predicting the motion of vascular branches during surgery. For example, Ambrosini et al. used a hidden Markov model (HMM) to track the catheter tip, enabling 3D blood vessels to be superimposed on each frame of an X-ray fluorescence image sequence during free breathing. Ma et al. introduced a new dynamic coronary artery prediction method based on Bayesian filtering and catheter tip tracking. They used electrocardiogram (ECG) data obtained from percutaneous coronary intervention (PCI) surgery to establish the correspondence between the catheter tip between image frames under different motion states and dynamically predict the coronary artery roadmap in subsequent image frames. Vernikouskaya et al. used a convolutional neural network (CNN) to learn the displacement of the catheter tip and associate the vascular motion with the respiratory motion predicted by CNN through ECG data.

[0007] However, the above methods all have the disadvantage of being unstable in predicting the location of hepatic blood vessels during intraoperative free breathing of patients. Summary of the Invention

[0008] In view of the above problems, the present invention provides a respiration compensation method, apparatus, device, and storage medium for overcoming or at least partially resolving the above problems. The method can accurately predict the current position and morphology of blood vessels in the presence of intraoperative respiration and an interventional guidewire, without contrast agent.

[0009] The present invention provides the following solutions:

[0010] A breathing compensation method, comprising:

[0011] Acquire a sequence of continuous frame X-ray images of the surgical area during surgery;

[0012] Extracting and obtaining a plurality of continuous liver mask images from the X-ray image sequence;

[0013] Calculating the position of the liver centroid in each of the liver mask images respectively;

[0014] Calculating the relative displacement between frames using the position of the liver mass center corresponding to each of the liver mask images, so as to calculate and fit a respiratory curve;

[0015] Dynamically predicting the vascular posture in each frame of the X-ray image sequence using the respiratory curve in combination with the initial vascular roadmap to obtain a plurality of rigid motion compensated vascular roadmaps;

[0016] Refining the guidewire in the blood vessel lumen of each X-ray image in the X-ray image sequence into a plurality of trajectory points, and using Bezier curve estimation to implement bending energy modeling to obtain energy characteristics of each trajectory point;

[0017] Screening out a number of control points for guidewire bending energy sampling in each X-ray image according to the energy characteristics of each trajectory point;

[0018] matching the control points corresponding to each of the X-ray images;

[0019] The matched control points are used to guide the rigid motion compensated blood vessel roadmaps to form a deformation field, thereby achieving elastic registration of image regions at different times, completing non-rigid motion compensation of blood vessels, and obtaining continuous blood vessel prediction results.

[0020] Preferably, a U-Net-based segmentation network is used to extract a plurality of continuous liver mask images from the X-ray image sequence; and a loss function defined according to the Dice similarity coefficient is used during the training process of the U-Net-based segmentation network.

[0021] Preferably, the respiratory curve is obtained by calculating and fitting the following formula:

[0022]

[0023] Where: Indicates the respiratory movement along the head and feet direction, Indicates the breathing movement along the left and right direction, I m (x,y) represents the pixel grayscale of the liver mask image, x,y represent the row coordinates and column coordinates of the image respectively, size represents the image scale, Indicates that x is the sum of the product of the row pixel coordinates and the corresponding pixel grayscale, Indicates that y is the sum of the products of the column pixel coordinates and the corresponding pixel grayscale.

[0024] Preferably, a dynamic time planning algorithm is used to match the control points at different times.

[0025] Preferably, using the matched control points to guide the rigid motion compensated vascular roadmaps to form a deformation field comprises:

[0026] Based on B-spline interpolation, a bending energy operator is introduced to perform deformation according to the maximum energy feature in the calculated area of ​​the interpolation control points supporting the deformation.

[0027] Preferably, given a fixed 2D image F with internal coordinates θ = (x1, y1) and image intensity f = F(θ), and a moving image M with corresponding coordinates φ = (x2, y2) and image intensity m = M(φ), the two images are registered using the following loss function:

[0028]

[0029] Ψ(f, m) = λS(f, m) + (1 - λ)Ψ(f, m - 1) where Ψ(f, m) denotes, λ denotes a weight, and S denotes a smoothing degree.

[0030] Preferably, the smoothing degree S is represented by the following formula:

[0031]

[0032] where v denotes a dense vector field defined for each pixel θ ∈ Ω.

[0033] A respiratory compensation device for performing the respiratory compensation method described above, the device comprising:

[0034] an image sequence acquisition unit configured to acquire a sequence of continuous frame X-ray images of a surgical region during surgery;

[0035] a liver mask image extraction unit configured to extract a plurality of continuous liver mask images from the sequence of X-ray images;

[0036] a liver centroid position calculation unit configured to calculate the position of the liver centroid in each of the liver mask images, respectively;

[0037] a respiratory curve fitting unit configured to calculate the relative displacement motion between frames using the position of the liver centroid corresponding to each of the liver mask images, respectively, in order to calculate and fit a respiratory curve;

[0038] a rigid motion compensation unit configured to dynamically predict the blood vessel pose in each frame of the sequence of X-ray images using the respiratory curve in combination with an initial blood vessel roadmap to obtain a plurality of rigid motion compensated blood vessel roadmaps;

[0039] a guide wire trajectory point extraction unit configured to respectively refine the guide wire within the blood vessel lumen of each X-ray image of the sequence of X-ray images into a plurality of trajectory points, and to achieve energy modeling of the curvature using Bezier curve estimation to obtain energy features of each of the trajectory points;

[0040] a control point screening unit configured to screen a plurality of control points of the guide wire curvature energy sampling in each of the X-ray images according to the energy features of each of the trajectory points;

[0041] a matching unit configured to match the control points corresponding to each of the X-ray images;

[0042] a non-rigid motion compensation unit configured to guide the plurality of rigid motion compensated blood vessel roadmaps to form a deformation field using the matched plurality of control points, to achieve elastic registration of the image region at different time instants, and to complete non-rigid motion compensation of the blood vessels to obtain continuous blood vessel prediction results.

[0043] A breathing compensation device, comprising a processor and a memory:

[0044] The memory is used to store program code and transmit the program code to the processor;

[0045] The processor is configured to execute the above-mentioned breathing compensation method according to instructions in the program code.

[0046] A computer-readable storage medium is used to store program codes, and the program codes are used to execute the above-mentioned breathing compensation method.

[0047] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0048] The embodiments of the present application provide a respiratory compensation method, device, equipment and storage medium, which decomposes respiratory motion into rigid displacement motion and non-rigid deformation motion, and compensates and corrects the characteristics of these two motions respectively. For rigid motion, the blood vessels are wrapped by the liver parenchyma, and the rigid motion of the blood vessels can be replaced by the liver. By tracking the centroid trajectory of the liver area in the X-ray image sequence, the motion trajectory curve of the blood vessels can be quickly obtained, and the initial blood vessel roadmap can be driven. Next, for the deformation correction of the blood vessels, the bending energy of the guide wire in the lumen of the blood vessel is calculated, and the structural features of the higher energy part are screened out to drive the elastic alignment of the image area at different times, thereby completing the non-rigid deformation compensation of the blood vessels. This method can effectively and quickly continuously track surgical instruments in blood vessels and reduce dependence on contrast agents.

[0049] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be derived from these drawings without inventive effort.

[0051] Figure 1 is a flow chart of a breathing compensation method provided by an embodiment of the present invention;

[0052] Figure 2 This is a framework diagram of a breathing compensation method implementation process provided by an embodiment of the present invention;

[0053] Figure 3 is a schematic diagram of a breathing compensation device provided by an embodiment of the present invention;

[0054] Figure 4 Schematic diagram of a breathing compensation device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0055] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of the present invention.

[0056] See also Figure 1 , which is provided by an embodiment of the present invention, such as Figure 1 As shown, the method may include:

[0057] S101: Acquire a sequence of continuous frame X-ray images of the surgical area during surgery;

[0058] S102: Extracting a plurality of consecutive liver mask images from the X-ray image sequence. In a specific implementation, the embodiment of the present application may provide a U-Net-based segmentation network for extracting a plurality of consecutive liver mask images from the X-ray image sequence. A loss function defined according to a Dice similarity coefficient is used during training of the U-Net-based segmentation network.

[0059] S103: Calculating the position of the liver centroid in each of the liver mask images respectively;

[0060] S104: Calculate the relative displacement between frames using the position of the liver mass center corresponding to each of the liver mask images, so as to calculate and fit a respiratory curve. In a specific implementation, the embodiment of the present application can provide a respiratory curve calculated and fitted using the following formula:

[0061]

[0062] Where: Indicates the respiratory movement along the head and feet direction, Indicates the breathing movement along the left and right direction, I m (x,y) represents the pixel grayscale of the liver mask image, x,y represent the row coordinates and column coordinates of the image respectively, size represents the image scale, Indicates that x is the sum of the product of the row pixel coordinates and the corresponding pixel grayscale, Indicates that y is the sum of the products of the column pixel coordinates and the corresponding pixel grayscale.

[0063] S105: dynamically predicting the blood vessel pose in each frame of the X-ray image sequence by using the breathing curve in combination with the initial blood vessel roadmap to obtain a plurality of rigid motion compensated blood vessel roadmaps;

[0064] S106: refining the guide wire in the blood vessel lumen of each X-ray image in the X-ray image sequence into a plurality of trajectory points, and achieving energy modeling of each trajectory point by using a Bezier curve estimation to obtain an energy feature of each trajectory point;

[0065] S107: screening a plurality of control points of the guide wire bending energy sampling in each X-ray image according to the energy feature of each trajectory point;

[0066] S108: matching the control points corresponding to each X-ray image; in specific implementation, the embodiment of the application can provide an algorithm of dynamic time planning to match the control points at different times.

[0067] S109: guiding a plurality of the rigid motion compensated blood vessel roadmaps to form a deformation field by using the matched control points, realizing elastic registration of image regions at different times, and completing non-rigid motion compensation of the blood vessels to obtain a continuous blood vessel prediction result.

[0068] In specific implementation, the embodiment of the application can provide that a bending energy operator is introduced on the basis of B-spline interpolation, and the maximum energy feature in the region is calculated according to the interpolation control points supporting deformation to perform deformation.

[0069] Further, given a fixed two-dimensional image F, the image coordinates are θ=(x1, y1), and the image intensity is f=F(θ); a moving image M is provided, and the corresponding coordinates are φ=(x2, y2), and the image intensity is m=M(φ); the two images are matched by the following loss function:

[0070]

[0071] In the formula, Ψ(f, m) represents, λ represents a weight, and S represents smoothness.

[0072] The smoothness S is represented by the following formula:

[0073]

[0074] In the formula, v represents a dense vector field defined for each pixel θ∈Ω.

[0075] The respiratory compensation method provided in the embodiment of the present application uses adaptive tracking and bending energy modeling to achieve stable vascular prediction under free breathing. First, a rigid displacement compensation method between image frames based on image domain conversion and adaptive centroid tracking is introduced. It fits the respiratory curve from the input X-ray image sequence to provide a temporal motion prior for aligning the vascular roadmap between image frames. Secondly, a compensation method based on bending energy modeling is proposed to correct respiratory deformation, using the energy characteristics of the guidewire to drive non-rigid alignment. The control points sampled according to the bending energy guide the local image to form a deformation field, which helps to dynamically superimpose the deformed vascular roadmap on the X-ray image sequence. The effective and rapid compensation that can be achieved by this method has the potential to improve liver intervention and reduce dependence on contrast agents.

[0076] The method provided in the embodiments of the present application is described in detail below.

[0077] In order to complete the respiratory compensation on the X-ray image sequence and realize the continuous prediction of the liver blood vessels, such as Figure 2 As shown in the figure, the framework of the method constructed in the embodiment of the present application mainly includes two parts. This framework extracts the liver region from the input X-ray image and adaptively tracks the center of mass position across multiple frames. By calculating the relative displacement of the center of mass between frames, the respiratory motion curve is calculated and fitted, thereby achieving dynamic vascular position prediction.

[0078] It then performs non-rigid deformation compensation by modeling the guidewire's bending energy. After calculating the bending energy of the guidewire structure, it selects energy-featured control points to guide non-rigid registration and generates a deformation field around the guidewire based on local image intensity variations, ultimately achieving continuous prediction of vascular morphology.

[0079] 1. Rigid motion compensation based on adaptive center of mass tracking.

[0080] The rapid calculation of rigid displacement between image frames relies on the tracking of image centroids, but the accuracy of this process is easily affected by the position of the diaphragm and the quality of X-ray imaging. In order to dynamically track the centroid in X-ray image-guided liver intervention, the liver region is first extracted. We obtain the liver mask from the X-ray image sequence through the network segmentation method. In order to obtain continuous liver segmentation results from the input X-ray image sequence, we train a U-Net-based segmentation network. During the training process, we obtain the gold standard liver segmentation results from doctors, and at the same time improve the intensity difference between the liver region and the background region through image preprocessing. Our loss function is defined based on the Dice similarity coefficient:

[0081]

[0082] Among them B P and BGT Represent the predicted results and segmentation gold standard of the liver, respectively, and they have the same scale as the input X-ray image.

[0083] After obtaining continuous liver mask images I m (x, y), where m represents the respiratory motion within a respiratory cycle. Each mask image is a binary image defined on a regular two-dimensional grid represented by the xy coordinate system. Based on this, we extract the position of the liver center of mass to calculate the relative displacement between frames and use their position trajectory to fit the respiratory curve. The formula is as follows:

[0084]

[0085]

[0086] Where: Indicates the respiratory movement along the head and feet direction, Indicates the breathing movement along the left and right direction, I m (x,y) represents the pixel grayscale of the liver mask image, x,y represent the row coordinates and column coordinates of the image respectively, size represents the image scale, Indicates that x is the sum of the product of the row pixel coordinates and the corresponding pixel grayscale, Indicates that y is the sum of the products of the column pixel coordinates and the corresponding pixel grayscale.

[0087] In order to achieve continuous respiratory motion compensation, it is necessary to calculate the relative motion between two adjacent frames within a respiratory cycle. The formula is as follows:

[0088]

[0089] in and Represents two adjacent moments.

[0090] In this process, displacement compensation is equivalent to applying an affine transformation to drive the vascular roadmap to move. Each element in the affine transformation matrix can be used and Finally, this rigid transformation matrix will be applied as the initial road map and dynamic blood vessel prediction results will be obtained in subsequent image frames.

[0091] In clinical practice, an initial vascular roadmap is established in the first frame through 3D / 2D registration. Respiration compensation is then performed to adjust the position and posture of the vascular roadmap in subsequent frames. The essence of compensation lies in guiding the movement and deformation of the initial roadmap by modeling intraoperative respiration, which includes both rigid vascular displacement and non-rigid vascular deformation.

[0092] 2. Non-rigid motion compensation based on bending energy modeling.

[0093] After rigid displacement compensation, dynamic vascular prediction is achieved. However, the nonparametric deformation between the liver parenchyma and intrahepatic vessels during respiration still affects the accuracy of the vascular roadmap prediction. To improve accuracy and computational efficiency, we utilize liver region images from guidewires and continuous X-ray images to estimate the nonrigid deformation field.

[0094] As the guidewire advances along the lumen of a vascular branch, it bends, storing mechanical energy in the material. The bending elasticity of the structure creates a local maximum entropy (LME), which makes the curved structure more easily deformable during breathing, and the residual area will translate with the respiratory trend.

[0095] Therefore, the embodiment of the present application refines the guidewire into trajectory points and uses the curvature of each point to characterize the bending energy. The bending energy modeling is realized by using Bezier curve estimation to obtain the energy characteristics of each point on the guidewire, and the control point P is obtained by screening. i,j , then the matching between the control points at different times needs to be established. For each set of trajectory points that need to be matched, the dynamic time planning (DTW) algorithm is used to achieve the matching. The corresponding point sets are represented as s i ={p1,p2,…,p m} and s j ={q1,q2,…,q n}.

[0096] The purpose of DTW is to find a matching path in the matrix grid and point p in two sequences i and dot q j When the continuity and monotonicity constraints are met, multiple paths can be obtained, where the formula for minimizing the regularization cost is:

[0097]

[0098] By recursively calculating the path D(i,j), continuous matching of two point set sequences is obtained.

[0099] Next, these matched feature control points are used to guide non-rigid registration. Here, a bending energy operator can be introduced based on B-spline interpolation. It calculates the maximum energy feature within the region based on the interpolation control points that support the deformation to perform deformation. Given a fixed two-dimensional image F with image coordinates θ = (x1, y1) and image intensity f = F(θ), then the moving image M also has corresponding coordinates φ = (x2, y2) and image intensity m = M(φ). The two images can be registered using the following loss function:

[0100] C=∑ T(θ)∈ΩΨ(f,m) + λS, (6)

[0101] The above equation can be optimized by a similarity measure Ψ that evaluates the coordinate mapping M(θ) = θ + v, where v is a dense vector field defined for each pixel θ ∈ Ω, assuming it enables a one-to-one mapping from F to M. The smoothness S of v is added to C with a weight λ, driving M to reach a meaningful coordinate map.

[0102] The vector field v here is defined by control points P i,j = p x , p y = p x = [(p x , 0, 0), …, (p x , I, J)] and p y = [(p y , 0, 0), …, (p y , I, J)] are defined for n = I x J control points, with pixel spacing r = r x , r y .

[0103] The smoothness S can be represented by the following equation:

[0104]

[0105] where S is used to smooth the gradient variation in the deformation process, and the neighborhood within the local control range of the control point P i,j is smoothly deformed to obtain the deformed continuous blood vessel prediction result in the subsequent X-ray image.

[0106] In summary, the respiratory compensation method provided by the present application decomposes the respiratory motion into rigid displacement motion and non-rigid deformation motion, and compensates and corrects the two motions respectively. For the rigid motion, the blood vessel is wrapped by the liver parenchyma, and the rigid motion of the blood vessel can be replaced by the liver. The centroid trajectory of the liver region in the X image sequence can be tracked to quickly obtain the motion trajectory curve of the blood vessel, and the initial blood vessel roadmap is driven by the motion trajectory curve. Next, for the deformation correction of the blood vessel, the bending energy of the guide wire in the blood vessel lumen is calculated, and the structural features with high energy are selected to drive the elastic registration of the image region at different times, and the non-rigid deformation compensation of the blood vessel is completed. The method can effectively and quickly track the surgical instrument in the blood vessel, and reduce the dependence on contrast agents.

[0107] Referring to Figure 3 , the embodiments of the present application can also provide a respiratory compensation device, as shown in Figure 3 , which can include:

[0108] An image sequence acquisition unit 301 is used to acquire a sequence of continuous frame X-ray images of the surgical area during surgery;

[0109] A liver mask image extraction unit 302 is configured to extract a plurality of consecutive liver mask images from the X-ray image sequence;

[0110] a liver mass center position calculation unit 303, configured to calculate the position of the liver mass center in each of the liver mask images;

[0111] A respiratory curve fitting unit 304 is configured to calculate the relative displacement between frames using the position of the liver mass center corresponding to each of the liver mask images, so as to calculate and fit a respiratory curve;

[0112] A rigid motion compensation unit 305 is configured to dynamically predict the blood vessel posture in each frame of the X-ray image sequence using the respiratory curve in combination with the initial blood vessel roadmap to obtain a plurality of rigid motion compensated blood vessel roadmaps;

[0113] a guidewire trajectory point extraction unit 306 for refining the guidewire in the vascular lumen of each X-ray image in the X-ray image sequence into a plurality of trajectory points, and obtaining an energy feature of each trajectory point by implementing bending energy modeling using Bezier curve estimation;

[0114] A control point screening unit 307 is configured to screen a number of control points for guidewire bending energy sampling in each X-ray image according to the energy characteristics of each trajectory point;

[0115] a matching unit 308, configured to match the control points corresponding to the X-ray images;

[0116] The non-rigid motion compensation unit 309 is used to use the matched control points to guide the rigid motion compensated vascular roadmaps to form a deformation field, achieve elastic registration of image areas at different times, complete non-rigid motion compensation of blood vessels, and obtain continuous blood vessel prediction results.

[0117] An embodiment of the present application may further provide a breathing compensation device, the device comprising a processor and a memory:

[0118] The memory is used to store program code and transmit the program code to the processor;

[0119] The processor is configured to execute the steps of the above-mentioned breathing compensation method according to the instructions in the program code.

[0120] like Figure 4As shown, a breathing compensation device provided by an embodiment of the present application may include: a processor 10, a memory 11, a communication interface 12, and a communication bus 13. The processor 10, the memory 11, and the communication interface 12 all communicate with each other via the communication bus 13.

[0121] In the embodiment of the present application, the processor 10 may be a central processing unit (CPU), an application specific integrated circuit, a digital signal processor, a field programmable gate array, or other programmable logic devices.

[0122] The processor 10 may call a program stored in the memory 11 . Specifically, the processor 10 may execute operations in an embodiment of the breathing compensation method.

[0123] The memory 11 is used to store one or more programs. The program may include program code, and the program code includes computer operating instructions. In the embodiment of the present application, the memory 11 stores at least a program for implementing the following functions:

[0124] Acquire a sequence of continuous frame X-ray images of the surgical area during surgery;

[0125] Extracting and obtaining a plurality of continuous liver mask images from the X-ray image sequence;

[0126] Calculating the position of the liver centroid in each of the liver mask images respectively;

[0127] Calculating the relative displacement between frames using the position of the liver mass center corresponding to each of the liver mask images, so as to calculate and fit a respiratory curve;

[0128] Dynamically predicting the vascular posture in each frame of the X-ray image sequence using the respiratory curve in combination with the initial vascular roadmap to obtain a plurality of rigid motion compensated vascular roadmaps;

[0129] Refining the guidewire in the blood vessel lumen of each X-ray image in the X-ray image sequence into a plurality of trajectory points, and using Bezier curve estimation to implement bending energy modeling to obtain energy characteristics of each trajectory point;

[0130] Screening out a number of control points for guidewire bending energy sampling in each X-ray image according to the energy characteristics of each trajectory point;

[0131] Matching the control points corresponding to each of the X-ray images;

[0132] The matched control points are used to guide the rigid motion compensated blood vessel roadmaps to form a deformation field, thereby achieving elastic registration of image regions at different times, completing non-rigid motion compensation of blood vessels, and obtaining continuous blood vessel prediction results.

[0133] In one possible implementation, the memory 11 may include a program storage area and a data storage area, wherein the program storage area can store an operating system and application programs required for at least one function (such as a file creation function, a data reading and writing function), etc.; the data storage area can store data created during use, such as initialization data, etc.

[0134] In addition, the memory 11 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device.

[0135] The communication interface 12 may be an interface of a communication module, and is used to connect to other devices or systems.

[0136] Of course, it needs to be explained that Figure 4 The structure shown does not constitute a limitation on the respiratory compensation device in the embodiment of the present application. In actual application, the respiratory compensation device may include Figure 4 More or fewer components than shown, or combinations of certain components.

[0137] The embodiment of the present application may further provide a computer-readable storage medium, wherein the computer-readable storage medium is used to store program code, and the program code is used to execute the steps of the above-mentioned breathing compensation method.

[0138] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0139] Through the description of the above embodiments, it can be seen that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments of the present application.

[0140] Each embodiment in this specification is described in a progressive manner. The same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple. For relevant parts, refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without expending creative work.

[0141] The above description is only a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention are included in the scope of protection of the present invention.

Claims

1. A breathing compensation method, characterized in that: include: Acquire a sequence of continuous frame X-ray images of the surgical area during surgery; Extracting and obtaining a plurality of continuous liver mask images from the X-ray image sequence; Calculating the position of the liver centroid in each of the liver mask images respectively; Calculating the relative displacement between frames using the position of the liver mass center corresponding to each of the liver mask images, so as to calculate and fit a respiratory curve; Dynamically predicting the vascular posture in each frame of the X-ray image sequence using the respiratory curve in combination with the initial vascular roadmap to obtain a plurality of rigid motion compensated vascular roadmaps; Refining the guidewire in the blood vessel lumen of each X-ray image in the X-ray image sequence into a plurality of trajectory points, and using Bezier curve estimation to implement bending energy modeling to obtain energy characteristics of each trajectory point; Screening out a number of control points for guidewire bending energy sampling in each X-ray image according to the energy characteristics of each trajectory point; matching the control points corresponding to each of the X-ray images; The matched control points are used to guide the rigid motion compensated blood vessel roadmaps to form a deformation field, thereby achieving elastic registration of image regions at different times, completing non-rigid motion compensation of blood vessels, and obtaining continuous blood vessel prediction results.

2. The breathing compensation method according to claim 1, characterized in that: A U-Net-based segmentation network is used to extract a plurality of continuous liver mask images from the X-ray image sequence; a loss function defined according to the Dice similarity coefficient is used in the training process of the U-Net-based segmentation network.

3. The breathing compensation method according to claim 1, wherein: The respiratory curve is obtained by calculation and fitting using the following formula: Where: Indicates the respiratory movement along the head and feet direction, Indicates the breathing movement along the left and right direction, represents the pixel grayscale of the liver mask image, , Represent the row coordinates and column coordinates of the image respectively, represents the image scale, express Take the sum of the products of row pixel coordinates and corresponding pixel grayscale, express Take the sum of the products of the column pixel coordinates and the corresponding pixel grayscale.

4. The breathing compensation method according to claim 1, wherein: A dynamic time planning algorithm is used to match the control points at different times.

5. The breathing compensation method according to claim 1, characterized in that: Using the matched control points to guide the rigid motion compensated vascular roadmaps to form a deformation field includes: Based on B-spline interpolation, a bending energy operator is introduced to perform deformation according to the maximum energy feature in the calculated area of ​​the interpolation control points supporting the deformation.

6. The breathing compensation method according to claim 5, characterized in that: Given a fixed 2D image , the image internal coordinates are , the image intensity is ; Moving images , and the corresponding coordinates and image intensity , the two images are registered using the following loss function: Where: represents the weight, Indicates smoothness.

7. The breathing compensation method according to claim 6, characterized in that: Smoothness It is expressed by the following formula: Where: For each pixel A dense vector field defined by .

8. A breathing compensation device, characterized in that: For performing the breathing compensation method according to any one of claims 1 to 7, the device comprises: An image sequence acquisition unit, used for acquiring a sequence of continuous-frame X-ray images of the surgical area during surgery; a liver mask image extraction unit, configured to extract a plurality of consecutive liver mask images from the X-ray image sequence; a liver mass center position calculation unit, configured to calculate the position of the liver mass center in each of the liver mask images; a respiratory curve fitting unit, configured to calculate the relative displacement between frames using the position of the liver mass center corresponding to each of the liver mask images, so as to calculate and fit a respiratory curve; a rigid motion compensation unit, configured to dynamically predict the blood vessel posture in each frame of the X-ray image sequence by using the respiratory curve in combination with an initial blood vessel roadmap to obtain a plurality of rigid motion compensated blood vessel roadmaps; a guidewire trajectory point extraction unit, configured to refine the guidewire in the vascular lumen of each X-ray image in the X-ray image sequence into a plurality of trajectory points, and obtain an energy feature of each of the trajectory points by implementing bending energy modeling using Bezier curve estimation; a control point screening unit, configured to screen a number of control points for sampling the guide wire bending energy in each of the X-ray images according to the energy characteristics of each of the trajectory points; a matching unit, configured to match the control points corresponding to the respective X-ray images; The non-rigid motion compensation unit is used to use the matched control points to guide the rigid motion compensated vascular roadmaps to form a deformation field, realize elastic registration of image areas at different times, complete the non-rigid motion compensation of the blood vessels, and obtain continuous blood vessel prediction results.

9. A breathing compensation device, characterized in that The device includes a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is configured to execute the breathing compensation method according to any one of claims 1 to 7 according to instructions in the program code.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store program code, and the program code is used to execute the breathing compensation method according to any one of claims 1 to 7.

Images