An image registration apparatus and method

Abstract

The application discloses an image registration device and method, which specifically determines three-dimensional images of multiple intermediate phases between a first phase and a second phase according to a first-phase three-dimensional image under target site expansion and a second-phase three-dimensional image under target site contraction, adjusts pose parameters of a virtual emission source, determines projection two-dimensional images obtained by projecting three-dimensional images of each phase under multiple emission source pose parameters, obtains a projection two-dimensional image set, determines target virtual emission source pose parameters of target two-dimensional images matched with real two-dimensional images in the projection two-dimensional image set, projects a target object marked in the first-phase three-dimensional image to the real two-dimensional image by using the target virtual emission source pose parameters, and obtains a registration image. The scheme can register the target object in the real three-dimensional image by only shooting two three-dimensional images, does not need to expose the patient to rays for a long time to shoot a 4DCT image, and can reduce radiation received by the patient.

Description

Technical Field

[0001] This application relates to the field of intelligent surgical system technology, and in particular to an image registration device and method. Background Technology

[0002] The respiration and heartbeat of living organisms such as humans and animals are usually accompanied by the regular contraction and relaxation of cavities, which makes it difficult to perform precise surgical procedures. For example, in many surgical procedures such as percutaneous puncture, transoral puncture, and laparoscopic surgery, the position of the target subject is greatly moved due to the influence of respiration, resulting in a decrease in the success rate of the surgery.

[0003] To overcome the impact of respiration on surgical accuracy, current techniques typically require obtaining images of the complete respiratory phase using 4DCT technology before surgery. Then, the images captured during surgery are registered with the 4DCT images to find a respiratory phase image at a specific moment that matches the intraoperative images. The location of the target object is determined based on this respiratory phase image, and the surgical procedure is then performed based on that location.

[0004] 4DCT technology is an image processing technique used for image reconstruction. It can provide image sequences in the temporal dimension and high-resolution images in the spatial dimension. The basic principle of 4DCT technology is to use a CT scanner to perform multiple scans, acquiring images of the patient at different time points in each scan. By superimposing and processing these images from different time points, a dynamic image sequence containing temporal information is obtained. This dynamic image sequence is called a 4DCT image.

[0005] Therefore, it can be concluded that 4D CT surgery requires patients to undergo CT scans for an extended period. The X-rays and other radiation used in CT scans generate significant amounts of radiation, and prolonged, unprotected exposure to this radiation can be quite harmful to the human body. Summary of the Invention

[0006] The purpose of this specification is to provide an image registration device and method to solve the problem that the use of 4DCT technology in existing surgeries causes significant harm to the human body.

[0007] To address the aforementioned technical problems, the present invention provides an image registration apparatus, comprising: a first acquisition unit, configured to acquire a first-phase three-dimensional image of a target site under expansion and a second-phase three-dimensional image of a target site under contraction, and to mark a target object in the first-phase three-dimensional image; a calculation unit, configured to calculate the deformation field of the target site from the first phase to the second phase based on the first-phase three-dimensional image and the second-phase three-dimensional image; a first determination unit, configured to determine three-dimensional images of multiple phases between the first phase and the second phase based on the deformation field; a virtual projection unit, configured to adjust the pose parameters of a virtual emitter for the acquired three-dimensional images of each phase, and determine projected two-dimensional images obtained by projecting the three-dimensional images of each phase under multiple emitter position parameters, thereby obtaining a set of projected two-dimensional images; a second acquisition unit, configured to acquire a real two-dimensional image of the target site of the patient; a second determination unit, configured to determine the virtual emitter pose parameters of a target two-dimensional image in the set of projected two-dimensional images that matches the real two-dimensional image; and a registration unit, configured to project the target object marked in the first-phase three-dimensional image onto the real two-dimensional image using the virtual emitter pose parameters corresponding to the target two-dimensional image, thereby obtaining a registered image.

[0008] In some embodiments, the first determining unit includes: a first processing subunit, configured to multiply the deformation field by k / n respectively to obtain a plurality of n-1 interpolated deformation fields, wherein n is a natural number and k takes the value of each natural number in [1, n-1]; and a second processing subunit, configured to apply each interpolated deformation field to the three-dimensional image of the first phase respectively to obtain three-dimensional images of n-1 phases from the first phase to the second phase.

[0009] In some embodiments, the registration unit includes: a first determining subunit, configured to determine the projection matrix of the virtual transmitter based on the pose parameters of the virtual transmitter corresponding to the target two-dimensional image; a transformation subunit, configured to convert the first phase three-dimensional image into point cloud data of the target object while keeping the coordinate system unchanged; and a projection subunit, configured to project the point cloud data of the target object onto the real two-dimensional image using the projection matrix to obtain a registered image.

[0010] In some embodiments, the calculation unit includes repeatedly executing the following steps until the dissimilarity is less than a predetermined dissimilarity threshold, and then using the latest current deformation field as the deformation field of the target part from the first phase to the second phase: inputting the original images of the first phase three-dimensional image and the second phase three-dimensional image into an artificial intelligence neural network to obtain the current deformation field; applying the current deformation field to the first phase three-dimensional image to obtain the deformed first phase three-dimensional image; calculating the dissimilarity between the deformed first phase three-dimensional image and the original image of the first phase three-dimensional image; and adjusting the parameters of the artificial intelligence neural network if the dissimilarity is greater than or equal to the predetermined dissimilarity threshold.

[0011] In some embodiments, the second determining unit includes: a first calculation subunit, configured to calculate the similarity between each projected two-dimensional image in the projected two-dimensional image set and the real two-dimensional image respectively; and a first acquisition subunit, configured to take the projected two-dimensional image corresponding to the largest similarity value as the target two-dimensional image, and acquire the virtual emission source pose parameters corresponding to the target two-dimensional image.

[0012] In some embodiments, the apparatus further includes: a third determining unit, configured to determine the actual positional relationship between the transmitter of the imaging device used to capture a two-dimensional image of the patient site, the target patient site, and the imaging plate before adjusting the pose parameters of the virtual transmitter; and a fourth determining unit, configured to determine the virtual positional relationship between the virtual imaging plate, the first three-dimensional image, and the virtual transmitter based on the actual positional relationship; and to determine the pose parameter adjustment range of the virtual transmitter based on the virtual positional relationship, wherein the pose of the transmitter of the imaging device relative to the target patient site is located in the middle of the pose parameter adjustment range.

[0013] In some embodiments, the virtual projection unit includes: a search subunit, configured to use the pose of the imaging device's emission source relative to the patient's target site as the initial pose parameter of the virtual emission source, adjust the pose parameter of the virtual emission source starting from the initial pose parameter, and search for a target two-dimensional image; during the search for the target two-dimensional image, for each virtual emission source pose parameter determined, calculate the similarity between the projected two-dimensional image corresponding to the pose parameter and the real two-dimensional image, and if the similarity is greater than a predetermined similarity threshold, take the projected two-dimensional image corresponding to the largest similarity value as the target two-dimensional image, and obtain the virtual emission source pose parameter corresponding to the target two-dimensional image.

[0014] In some embodiments, the virtual projection unit further includes: a second acquisition subunit, configured to acquire the pose of the emission source relative to the patient target area recorded by the imaging device when capturing a real two-dimensional image; or, a third acquisition subunit, configured to acquire a first pose of the emission source and a second pose of the patient target area determined by the optical positioning system when capturing a real two-dimensional image; and a second calculation subunit, configured to determine the pose of the emission source of the imaging device relative to the patient target area based on the first pose and the second pose.

[0015] In some embodiments, the target site includes the lungs, the first phase three-dimensional image is an inspiratory three-dimensional image, and the second phase three-dimensional image is an expiratory three-dimensional image; or, the target site includes the heart or blood vessels, the first phase three-dimensional image is a diastolic three-dimensional image, and the second phase three-dimensional image is a systolic three-dimensional image.

[0016] A second aspect of this specification provides an image registration method, comprising: acquiring a first-phase three-dimensional image of a target site under expansion and a second-phase three-dimensional image of a target site under contraction, and marking a target object in the first-phase three-dimensional image; calculating the deformation field of the target site from the first phase to the second phase based on the first-phase three-dimensional image and the second-phase three-dimensional image; determining three-dimensional images of multiple phases between the first phase and the second phase based on the deformation field; adjusting the pose parameters of a virtual emitter for each acquired phase three-dimensional image, and determining projected two-dimensional images obtained by projecting the three-dimensional image of each phase under multiple emitter position parameters, thereby obtaining a set of projected two-dimensional images; acquiring a real two-dimensional image of the target site of the patient; determining the virtual emitter pose parameters of a target two-dimensional image in the set of projected two-dimensional images that match the real two-dimensional image; and projecting the target object marked in the first-phase three-dimensional image onto the real two-dimensional image using the virtual emitter pose parameters corresponding to the target two-dimensional image, thereby obtaining a registered image.

[0017] In some embodiments, determining a three-dimensional image of multiple phases between the first phase and the second phase based on the deformation field includes: multiplying the deformation field by k / n respectively to obtain multiple n-1 interpolated deformation fields, where n is a natural number and k takes the value of any natural number in [1, n-1]; applying each interpolated deformation field to the three-dimensional image of the first phase to obtain a three-dimensional image of n-1 phases between the first phase and the second phase.

[0018] In some embodiments, projecting the target object marked in the first phase three-dimensional image onto a real two-dimensional image using the virtual emitter pose parameters corresponding to the target two-dimensional image to obtain a registration image includes: determining the projection matrix of the virtual emitter based on the virtual emitter pose parameters corresponding to the target two-dimensional image; converting the first phase three-dimensional image into point cloud data of the target object while keeping the coordinate system unchanged; and projecting the point cloud data of the target object onto a real two-dimensional image using the projection matrix to obtain a registration image.

[0019] In some embodiments, calculating the deformation field of the target region from the first phase to the second phase based on the first phase three-dimensional image and the second phase three-dimensional image includes repeatedly executing the following steps until the dissimilarity is less than a predetermined dissimilarity threshold, and then using the latest current deformation field as the deformation field of the target region from the first phase to the second phase: inputting the original images of the first phase three-dimensional image and the second phase three-dimensional image into an artificial intelligence neural network to obtain the current deformation field; applying the current deformation field to the first phase three-dimensional image to obtain the deformed first phase three-dimensional image; calculating the dissimilarity between the deformed first phase three-dimensional image and the original image of the first phase three-dimensional image; and adjusting the parameters of the artificial intelligence neural network if the dissimilarity is greater than or equal to the predetermined dissimilarity threshold.

[0020] In some embodiments, determining the virtual emitter pose parameters of a target two-dimensional image that matches a real two-dimensional image in a set of projected two-dimensional images includes: calculating the similarity between each projected two-dimensional image in the set of projected two-dimensional images and the real two-dimensional image respectively; taking the projected two-dimensional image corresponding to the largest similarity value as the target two-dimensional image, and obtaining the virtual emitter pose parameters corresponding to the target two-dimensional image.

[0021] In some embodiments, before adjusting the pose parameters of the virtual transmitter, the method further includes: determining the actual positional relationship between the transmitter of the imaging device used to capture a two-dimensional image of the patient site, the target patient site, and the imaging plate; determining the virtual positional relationship between the virtual imaging plate, the first three-dimensional image, and the virtual transmitter based on the actual positional relationship; and determining the pose parameter adjustment range of the virtual transmitter based on the virtual positional relationship, wherein the pose of the transmitter of the imaging device relative to the target patient site is located in the middle of the pose parameter adjustment range.

[0022] In some embodiments, determining the virtual emitter pose parameters of a target two-dimensional image that matches a real two-dimensional image in a set of projected two-dimensional images includes: using the pose of the emitter of the imaging device relative to the target part of the patient as the initial pose parameter of the virtual emitter; adjusting the pose parameter of the virtual emitter starting from the initial pose parameter and searching for target two-dimensional images; during the search for target two-dimensional images, for each virtual emitter pose parameter determined, calculating the similarity between the projected two-dimensional image corresponding to the pose parameter and the real two-dimensional image; if the similarity is greater than a predetermined similarity threshold, taking the projected two-dimensional image with the largest similarity value as the target two-dimensional image, and obtaining the virtual emitter pose parameters corresponding to the target two-dimensional image.

[0023] In some embodiments, before using the pose of the imaging device's emission source relative to the patient's target area as the initial pose parameter of the virtual emission source, the method further includes: acquiring the pose of the emission source relative to the patient's target area recorded by the imaging device when capturing a real two-dimensional image; or, acquiring a first pose of the emission source and a second pose of the patient's target area determined by the optical positioning system when capturing a real two-dimensional image; and determining the pose of the imaging device's emission source relative to the patient's target area based on the first pose and the second pose.

[0024] In some embodiments, the target site includes the lungs, the first phase three-dimensional image is an inspiratory three-dimensional image, and the second phase three-dimensional image is an expiratory three-dimensional image; or, the target site includes the heart or blood vessels, the first phase three-dimensional image is a diastolic three-dimensional image, and the second phase three-dimensional image is a systolic three-dimensional image.

[0025] A third aspect of the present invention provides a surgical system comprising: an image registration device as described in any of the first aspects; an imaging device for capturing a real two-dimensional image of a target area of ​​a patient; and a surgical robot including a robotic arm with surgical instruments mounted at its end for performing surgical operations based on the image registration results.

[0026] In some embodiments, the surgical system further includes: a ventilator for intervening in the patient's breathing process; a controller for controlling the ventilator to pause and controlling the imaging device to capture a real two-dimensional image upon receiving a target operation instruction; the target operation instruction is used to display the position of the target object in the real two-dimensional image.

[0027] A fourth aspect of the present invention provides an electronic device, comprising: a memory and a processor, wherein the processor and the memory are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to implement the image registration method according to any one of the first aspects.

[0028] The image registration apparatus and method provided by the present invention determine three-dimensional images of multiple intermediate phases between the first and second phases based on a first-phase three-dimensional image under the condition of target expansion and a second-phase three-dimensional image under the condition of target contraction. The pose parameters of the virtual transmitter are adjusted to determine the projected two-dimensional images obtained by projecting the three-dimensional image of each phase under the pose parameters of the multiple transmitters, thereby obtaining a set of projected two-dimensional images. The pose parameters of the target virtual transmitter of the target two-dimensional image that matches the real two-dimensional image in the set of projected two-dimensional images are determined. The target virtual transmitter pose parameters are used to project the target object marked in the first-phase three-dimensional image onto the real two-dimensional image to obtain the registered image. This solution requires only two 3D images to register the target object within a real 3D image, eliminating the need for prolonged patient exposure to 4DCT imaging and reducing radiation exposure. It automatically matches and projects the target object's image onto the real 2D image, guiding surgeons to quickly, conveniently, and accurately determine the target object's location, shortening surgical time and improving efficiency and success rate. Considering the impact of the target area's regular expansion and contraction movements on the surgery, it further enhances the success rate. The entire image processing is automated, requiring no manual annotation, reducing manual operation, extending image processing time, and lowering the demands on human intervention. Furthermore, it eliminates the need for markers at the target area and respiratory monitoring equipment to determine the phase of the real 2D image matching, reducing the equipment cost of the surgical system. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0030] Figure 1 A flowchart of the image registration method provided by the present invention;

[0031] Figure 2 A schematic diagram of the image registration method provided by the present invention;

[0032] Figure 3 This is a schematic diagram of the inspiratory phase, expiratory phase, and image segmentation results;

[0033] Figure 4 A flowchart for calculating the deformation field of a target region from the first phase to the second phase;

[0034] Figure 5 A schematic diagram for calculating the deformation field of the target area from the first phase to the second phase;

[0035] Figure 6 A flowchart for determining a three-dimensional image of multiple phases from the first phase to the second phase based on the deformation field;

[0036] Figure 7 A schematic diagram of a three-dimensional image of multiple phases from the first phase to the second phase determined based on the deformation field;

[0037] Figure 8 This is a schematic diagram illustrating the projection of a 3D image using a virtual emission source.

[0038] Figure 9 A schematic diagram illustrating the process of obtaining two-dimensional projected images at different angles by adjusting the pose parameters of a virtual transmitter.

[0039] Figure 10 This is a schematic diagram of a two-dimensional projection image obtained by projecting the three-dimensional image of each phase in multiple phases under the position parameters of multiple emission sources.

[0040] Figure 11 A schematic diagram of an imaging device for Fluor images;

[0041] Figure 12 A schematic diagram of a lung registration image;

[0042] Figure 13 A schematic diagram of a process for determining a target two-dimensional image;

[0043] Figure 14 A schematic diagram of an image registration device;

[0044] Figure 15 This is a schematic diagram of the surgical system;

[0045] Figure 16 This is a schematic diagram of an electronic device. Detailed Implementation

[0046] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0047] This invention provides an image registration method that can address the problem of significant harm to the human body caused by 4DCT technology. The image registration method provided by this invention eliminates the need for 4DCT images; it only requires capturing two 3D images—one for expansion and one for contraction of the target area—to achieve preoperative planning. The target object marked during preoperative planning is then registered with the intraoperative image and projected onto the intraoperative image.

[0048] Figure 1 The flowchart is for the image registration method provided by the present invention. Figure 2 This is a schematic diagram of the image registration method provided by the present invention. Figure 2 In the diagram, the images within the dashed boxes shown in 1, 2, and 3 can be prepared before the operation, while the content shown in 4 and 5 can be performed or determined during the operation.

[0049] like Figure 1 and Figure 2 As shown, the image registration method provided by the present invention includes the following steps:

[0050] S10: Acquire a first-phase three-dimensional image of the patient's target site under expansion and a second-phase three-dimensional image of the target site under contraction, and mark the target object in the first-phase three-dimensional image.

[0051] The target area refers to the part of the body that undergoes regular expansion and contraction movements. For example, the target area can be the lungs, heart, arteries, veins, etc.

[0052] The first phase refers to the state of volume expansion of the target area, and correspondingly, the first phase three-dimensional image is a three-dimensional image of the target area in the state of volume expansion.

[0053] The second phase refers to the state of volume contraction of the target area, and correspondingly, the three-dimensional image of the second phase is a three-dimensional image of the target area under the state of volume contraction.

[0054] When the target site is the heart or blood vessels, the first phase is the diastolic phase, and the corresponding three-dimensional image of the first phase is a diastolic three-dimensional image; the second phase is the systolic phase, and the corresponding three-dimensional image of the second phase is a systolic three-dimensional image.

[0055] When the target site is the lungs, the first phase is the inspiratory phase, and the corresponding three-dimensional image of the first phase is the inspiratory phase three-dimensional image; the second phase is the expiratory phase, and the corresponding three-dimensional image of the second phase is the expiratory phase three-dimensional image.

[0056] Preferably, the first phase is the state when the target volume expands to its maximum, and the second phase is the state when the target volume contracts to its minimum. For example, such as... Figure 3 As shown, when the target site is the lungs, it is important to ensure the patient is in a state of maximum inspiration and maximum expiration, followed by CT scanning while holding their breath, to obtain... Figure 3 The left image shows the inspiratory phase image X0, and the middle image shows the expiratory phase image X1.

[0057] After acquiring the first and second phase 3D images, object recognition, segmentation, and surgical planning can be performed. During this process, target objects can be marked. These target objects can be the site of the surgery to be performed, such as a lung nodule requiring puncture; or they can be sites that do not require surgery but affect the success rate, such as the bronchus traversed in a bronchiotomy for lung nodule removal.

[0058] The target object can be segmented on either the first-phase 3D image or the second-phase 3D image. Since the target area is in a state of volume expansion in the first-phase 3D image, the structural distinctions of the various internal tissues are more obvious. Therefore, to improve the accuracy of surgical planning and target object segmentation, it is preferable to perform surgical planning and target object segmentation based on the first-phase 3D image.

[0059] For example, Figure 3 The middle right image is the result of surgical planning and segmentation based on the inspiratory phase image X0 shown in the left image, where the black dots represent the segmented lung nodules.

[0060] S20: Calculate the deformation field of the target part from the first phase to the second phase based on the first phase three-dimensional image and the second phase three-dimensional image.

[0061] In some embodiments, the same feature point (e.g., the location of the bronchial bifurcation point) can be identified from the first phase three-dimensional image and the deformation field of the target part from the first phase to the second phase can be determined based on the positional relationship of the feature point.

[0062] In other embodiments, the deformation field can be determined by comparing the differences in grayscale values ​​between images.

[0063] like Figure 4 As shown, the present invention provides a method for calculating the deformation field of a target region from the first phase to the second phase, comprising the following steps S21 to S26.

[0064] S21: Input the original images of the first phase 3D image and the second phase 3D image into the artificial intelligence neural network to obtain the training deformation field.

[0065] Artificial intelligence neural networks can specifically be deep neural networks, or other types of artificial intelligence neural networks.

[0066] S22: Apply the training deformation field to the first phase three-dimensional image to obtain the deformed first phase three-dimensional image.

[0067] S23: Calculate the dissimilarity between the deformed first-phase 3D image and the original first-phase 3D image.

[0068] For example, dissimilarity can be achieved using a loss function that measures the correlation between pixels. In the formula, x and y represent two images, and μ x μ x σ y σ y These represent the mean and standard deviation of the entire graph (x and y), respectively, or a loss function that measures structural similarity. In the formula μ x μ y σ x σ y σ xy C1 and C2 represent the local mean, standard deviation, and cross-covariance of x and y, respectively, with C1 and C2 being parameters, and mean representing the average value.

[0069] S24: Determine whether the dissimilarity is less than a predetermined dissimilarity threshold. If the determination result is yes, proceed to step S25; if the determination result is no, after proceeding to step S26, jump to step S21 to continue execution.

[0070] S25: End training and use the latest current deformation field as the deformation field of the target part from the first phase to the second phase.

[0071] S26: Adjust the parameters of the artificial intelligence neural network.

[0072] Figure 4 The method shown uses dissimilarity as a loss function for backpropagation, iterating the network until convergence.

[0073] Figure 5 This is a schematic diagram for calculating the deformation field of the target region from the first phase to the second phase. In this diagram, 1 represents the inspiratory phase image X0, 2 represents the expiratory phase image X1, 3 represents the artificial intelligence neural network, 4 represents the deformation field output by the artificial intelligence neural network, and 5 represents the deformed first-phase three-dimensional image obtained after applying the current deformation field to the first-phase three-dimensional image.

[0074] Besides artificial intelligence neural networks, traditional registration methods can also be used, such as the B-spline registration algorithm, which includes the following steps: 1. Obtain control points in the image, which can be done by calculating feature points or generating them using a grid; 2. Adjust the control points in the moving image and use B-spline deformation to obtain the registered image; 3. Continuously adjust the positions of the control points to optimize the similarity between the registered image and the target image. When the similarity meets the requirements, the movement vector of each control point is the deformation field.

[0075] Compared to traditional registration methods, the deformation field determined by artificial intelligence neural networks is more accurate.

[0076] S30: Determine a three-dimensional image of multiple phases from the first phase to the second phase based on the deformation field.

[0077] In some embodiments, step S30 may be: multiplying the deformation field by k / n respectively to obtain a plurality of n-1 interpolated deformation fields, where n is a natural number and k takes the value of any natural number in [1, n-1]; applying each interpolated deformation field to the three-dimensional image of the first phase to obtain three-dimensional images of n-1 phases from the first phase to the second phase. That is, step S30 may perform uniform interpolation between the images of the first phase and the second phase to obtain three-dimensional images of multiple phases.

[0078] Specifically, such as Figure 6 and Figure 7 As shown, step S30 may include the following steps:

[0079] S31: Initialize k = 1.

[0080] S32: Multiply the deformation field by k / n to obtain the kth interpolated deformation field.

[0081] S33: Apply the kth interpolation deformation field to the three-dimensional image of the first phase to obtain the three-dimensional image of the kth phase between the first and second phases;

[0082] S34: Use k+1 as the new k.

[0083] S35: Determine if k equals n. If the result is yes, end; if the result is no, jump to S32 to continue execution.

[0084] In other embodiments, non-uniform interpolation can be performed between the images of the first phase and the second phase to obtain three-dimensional images of multiple phases.

[0085] S40: For the three-dimensional images of each phase that have been acquired, adjust the pose parameters of the virtual emission source, determine the projected two-dimensional images obtained by projecting the three-dimensional image of each phase under the position parameters of multiple emission sources, and obtain a set of projected two-dimensional images.

[0086] In the process of real two-dimensional image imaging, a radiation source usually emits rays toward the object being imaged, and an imaging plate is placed behind the object (i.e., on the side of the object away from the radiation source) to display the real two-dimensional image.

[0087] Step S40 uses the target area in the three-dimensional image of each phase to simulate the real target area, and uses a virtual emission source to simulate the real emission source. Thus, the projected two-dimensional image obtained by projecting the three-dimensional image of each phase through the virtual emission source is equivalent to the real two-dimensional image of the patient's target area taken under the real imaging system.

[0088] Figure 8 This diagram illustrates how a virtual X-ray source projects a 3D image to obtain a projected image. This projection method, also known as ray projection, belongs to the Digitally Reconstructed Radiograph (DRR) technique. For example, this method uses a virtual X-ray source to emit X-rays that penetrate the 3D image, generating a virtual X-ray image on the projection plane (i.e., the virtual imaging plate). The 3D image values ​​traversed by each ray are calculated (e.g., superimposed) and mapped to that point on the projection plane, becoming the pixel value of the projected image. Once all rays have been projected, the projected image is obtained.

[0089] In actual surgery, the imaging angle of the 2D imaging device needs to be adjusted according to the surgical requirements, and is not static. In order to ensure that each 2D image during the operation can be matched with a projected 2D image, this solution obtains projected 2D images from multiple angles by adjusting the pose parameters of the virtual transmitter. Figure 9 A schematic diagram illustrating how to obtain two-dimensional projected images at different angles by adjusting the pose parameters of a virtual transmitter. Figure 10 This is a schematic diagram of a two-dimensional projection image obtained by projecting the three-dimensional image of each phase in multiple phases under the position parameters of multiple emission sources.

[0090] S50: Acquire a true two-dimensional image of the target area on the patient.

[0091] True two-dimensional images can be taken during surgery. For example, when a surgeon needs to know the location of a lesion, they can control the imaging equipment to take a true two-dimensional image. Multiple true two-dimensional images can be taken as needed by the surgeon.

[0092] Two-dimensional images can be captured using techniques harmless to the human body. For example, they can be Fluor images captured using a Fluor imaging device. Fluor images are fluorescent images, and the imaging process does not use radiation, so they are harmless to the human body. Therefore, multiple images can be taken during the surgical procedure. Figure 11 This is a schematic diagram of an imaging device for Fluor images.

[0093] S60: Determine the pose parameters of the virtual emitter of the target two-dimensional image that matches the real two-dimensional image in the set of projected two-dimensional images.

[0094] S70: Using the virtual emitter pose parameters corresponding to the target two-dimensional image, the target object marked in the first phase three-dimensional image is projected onto the real two-dimensional image to obtain a registration image.

[0095] In some embodiments, step S70 projects only the "target object" marked in the first three-dimensional image into the real two-dimensional image, and does not project any content other than the "target object" into the real two-dimensional image.

[0096] like Figure 2 As shown, the difference between the registered image and the real two-dimensional image is the presence of black dots, as indicated by the arrows. These black dots represent the segmented lung nodules.

[0097] It is important to note that although the lung nodule exists in the actual two-dimensional image, it is not marked. If the surgeon performs the operation directly based on the actual two-dimensional image, they will need to determine the location of the lung nodule "during" the operation based on experience. Identifying the location of the lung nodule solely based on the actual "two-dimensional" image places high demands on the surgeon. In addition, an issue that cannot be ignored is that the target area is in a state of regular expansion and contraction, and the interval between the expansion and contraction states is very short (for example, the time interval between one inhalation and one exhalation is 1-2 seconds, and even with ventilator-controlled apnea, the pause time can only be a few seconds and cannot be longer). This means that the surgeon needs to identify the location of the lung nodule and perform the operation in a shorter time than this time interval. Obviously, the time is too tight, which can easily lead to surgical errors.

[0098] The image registration method provided by this invention can project pre-segmented lung nodule images onto a real two-dimensional image to obtain a registered image. This allows surgeons to easily identify the location of the lung nodules based on the registered image, enabling them to quickly move surgical instruments to the location of the nodules to perform surgical procedures. For example, Figure 12 This is a schematic diagram of a lung registration image, where the target object indicated by the black dot represents the location of a lung nodule. This nodule is projected onto a real two-dimensional image using the registration method provided by this invention. With this registration image, surgeons can conveniently and quickly ascertain the location of the puncture needle and the lung nodule.

[0099] In some embodiments, the first phase 3D image can be binarized firstly, and then the binarized first phase 3D image can be projected onto a real 2D image to obtain a registered image. During binarization, the voxels of the target object are set to 1, and the voxels other than the target object are set to 0.

[0100] Alternatively, the first phase 3D image can be converted into point cloud data of the target object while keeping the coordinate system unchanged, and then the point cloud data of the target object can be projected onto the real 2D image to obtain the registration image.

[0101] Because the target object in the first 3D image is a three-dimensional structure, its projection onto the real 2D image results in a dense 2D image with darker colors in the center and lighter colors at the edges, making it difficult for surgeons to distinguish the edges of the target object. However, accurately identifying the edges of the target object during surgery is crucial for surgical success. The method described above, which involves binarizing the first 3D image or converting it into point cloud data before projecting it onto the real 2D image to obtain the registration image, makes the projected boundaries of the target object in the registration image clearer and easier to distinguish, thereby improving surgical efficiency and accuracy.

[0102] In some embodiments, step S70 may also project the first phase three-dimensional image onto the real two-dimensional image and render the color of the projected image to be different from the color of the real two-dimensional image or to have a certain transparency. That is, in the registration image, rendering is used to distinguish the projected image from the real two-dimensional image.

[0103] Since the first-phase 3D image is fixed and the target object is marked in it, the projected portion of the registered image also marks the target image. The surgeon can determine whether the patient's target site is in the first phase and whether the target object is in the predetermined position during surgical planning based on the difference between the actual image and the projected image in the registered image. The surgeon can pre-control the surgical instruments to move near the predetermined position, waiting for the target site to move regularly until the target object reaches the predetermined position before performing the surgical procedure. For example, in a puncture procedure, the surgeon determines the predetermined position of the lung nodule during inspiration based on the projected image in the registered image. During the expiratory phase, the puncture needle is pre-moved near this predetermined position, allowing for faster puncture as the lung nodule moves to the predetermined position during inspiration.

[0104] In other embodiments, step S70 may also begin at the initial stage of surgery by projecting the entire first three-dimensional image onto the real two-dimensional image to assist the surgeon in controlling the movement of surgical instruments to the vicinity of the predetermined position. After the surgical instruments have moved to the vicinity of the predetermined position, only the "target object" marked in the first three-dimensional image is projected onto the real two-dimensional image, while content other than the "target object" is not projected onto the real two-dimensional image. For a description of the specific projection method and its effects, please refer to the above description, which will not be repeated here.

[0105] The image registration method provided by this invention determines three-dimensional images of multiple intermediate phases between the first and second phases based on a first-phase three-dimensional image under the condition of target expansion and a second-phase three-dimensional image under the condition of target contraction. It then adjusts the pose parameters of the virtual transmitter to determine the projected two-dimensional images obtained by projecting the three-dimensional image of each phase under the pose parameters of the multiple transmitters, thus obtaining a set of projected two-dimensional images. Next, it determines the target virtual transmitter pose parameters of the target two-dimensional image that matches the real two-dimensional image in the set of projected two-dimensional images. Finally, it uses these target virtual transmitter pose parameters to project the target object marked in the first-phase three-dimensional image onto the real two-dimensional image, thus obtaining a registered image. This solution requires only two 3D images to register the target object within a real 3D image, eliminating the need for prolonged patient exposure to 4DCT imaging and reducing radiation exposure. It automatically matches and projects the target object's image onto the real 2D image, guiding surgeons to quickly, conveniently, and accurately determine the target object's location, shortening surgical time and improving efficiency and success rate. Considering the impact of the target area's regular expansion and contraction movements on the surgery, it further enhances the success rate. The entire image processing is automated, requiring no manual annotation, reducing manual operation, extending image processing time, and lowering the demands on human intervention. Furthermore, it eliminates the need for markers at the target area and respiratory monitoring equipment to determine the phase of the real 2D image matching, reducing the equipment cost of the surgical system.

[0106] In some embodiments, step S60 may calculate the similarity between each projected two-dimensional image in the projected two-dimensional image set and the real two-dimensional image; and take the projected two-dimensional image corresponding to the largest similarity value as the target two-dimensional image.

[0107] For example, for Figure 10 The three-dimensional images of each phase shown are projected onto multiple emitter position parameters to obtain projected two-dimensional images. Assuming there are n+1 different three-dimensional images of each phase (i.e., the number of three-dimensional images shown in the left column is 1+n), and adjusting the virtual emitter pose parameters yields m pose parameters (i.e., the number of projected two-dimensional images in each row to the right of the arrow is m), we can set the phase index i∈[0,n] and the virtual emitter pose parameter index j∈[1,m]. The projected two-dimensional image of the j-th pose parameter corresponding to the three-dimensional image of the i-th phase is denoted as... The process of determining the target two-dimensional image can be as follows: Figure 13 As shown.

[0108] like Figure 13 As shown, the method includes: traversing each phase index i and the virtual transmitter pose parameter index j; obtaining the corresponding projected two-dimensional image based on the phase index i and the virtual transmitter pose parameter index j; and calculating the similarity between the projected two-dimensional image and the real two-dimensional image. After iterating through i and j, a series of similarity scores are obtained. i∈[0,n], j∈[1,m]; obtain such that Maximum index i * and j * By tracing and outputting the data, the pose parameters of the emission source corresponding to the target 2D image can be determined.

[0109] Before adjusting the pose parameters of the virtual transmitter, the actual positional relationship between the transmitter of the imaging device used to capture two-dimensional images of the patient site, the target patient site, and the imaging plate can be determined; the virtual positional relationship between the virtual imaging plate, the first three-dimensional image, and the virtual transmitter can be determined based on the actual positional relationship; and the pose parameter adjustment range of the virtual transmitter can be determined based on the virtual positional relationship, wherein the pose of the transmitter of the imaging device relative to the target patient site is located in the middle of the pose parameter adjustment range.

[0110] Some imaging devices can record the pose of the emission source relative to the imaging target. In such cases, the pose of the emission source relative to the target area of ​​the patient, as recorded by the imaging device when capturing a real two-dimensional image, can be directly obtained. For example, after obtaining a real two-dimensional image during intraoperative scanning, the imaging device can be fixed, and the relative position (x, y, z) and emission direction (α, β, γ) of the emission source relative to the patient can be recorded.

[0111] In some cases, the imaging device itself cannot record the pose of the emission source relative to the imaging target. In such cases, the first pose of the emission source of the imaging device and the second pose of the target area of ​​the patient can be determined by the optical positioning system. The first pose and the second pose can be obtained directly from the optical positioning system, and the pose of the emission source of the imaging device relative to the target area of ​​the patient can be determined based on the first pose and the second pose.

[0112] The adjustment range of the virtual transmitter's pose parameters should be set as small as possible, while including the possible locations of the actual imaging device, in order to reduce the optimization time, that is, to reduce the time spent finding the virtual transmitter's pose parameters corresponding to the target two-dimensional image.

[0113] In some embodiments, step S60 may use the pose of the imaging device's emission source relative to the patient's target site as the initial pose parameter of the virtual emission source, adjust the pose parameter of the virtual emission source starting from the initial pose parameter, and search for the target two-dimensional image; during the search for the target two-dimensional image, for each virtual emission source pose parameter determined, calculate the similarity between the projected two-dimensional image corresponding to the pose parameter and the real two-dimensional image, and if the similarity is greater than a predetermined similarity threshold, take the projected two-dimensional image corresponding to the largest similarity value as the target two-dimensional image.

[0114] For example, real imaging equipment is typically adjusted to its optimal pose before surgery, which is the pose with the least adjustment required during surgery. Therefore, the initial pose parameters of the virtual transmitter can be determined based on the optimal pose of the real imaging equipment. Starting from these initial pose parameters, the pose parameters of the virtual transmitter are adjusted, and the target 2D image is searched.

[0115] By using the initial pose parameters, the pose parameters of the virtual emitter corresponding to the target 2D image can be obtained more quickly, reducing image registration time.

[0116] This invention provides an image registration apparatus that can be used to implement the above-described image registration method. For example... Figure 14 As shown, the device includes a first acquisition unit 10, a calculation unit 20, a first determination unit 30, a virtual projection unit 40, a second acquisition unit 50, a second determination unit 60, and a registration unit 70.

[0117] The first acquisition unit 10 is used to acquire a first-phase three-dimensional image of the patient's target site under expansion and a second-phase three-dimensional image of the target site under contraction, and to mark the target object in the first-phase three-dimensional image.

[0118] The calculation unit 20 is used to calculate the deformation field of the target part from the first phase to the second phase based on the first phase three-dimensional image and the second phase three-dimensional image.

[0119] The first determining unit 30 is used to determine a three-dimensional image of multiple phases from the first phase to the second phase based on the deformation field.

[0120] The virtual projection unit 40 is used to adjust the pose parameters of the virtual emission source for the three-dimensional images of each phase that have been acquired, and to determine the projected two-dimensional images obtained by projecting the three-dimensional images of each phase under the position parameters of multiple emission sources, so as to obtain a set of projected two-dimensional images.

[0121] The second acquisition unit 50 is used to acquire a real two-dimensional image of the patient's target area.

[0122] The second determining unit 60 is used to determine the virtual emission source pose parameters of the target two-dimensional image that matches the real two-dimensional image in the set of projected two-dimensional images.

[0123] The registration unit 70 is used to project the target object marked in the first phase three-dimensional image onto the real two-dimensional image using the virtual emission source pose parameters corresponding to the target two-dimensional image, so as to obtain a registration image.

[0124] In some embodiments, the first determining unit includes: a first processing subunit, configured to multiply the deformation field by k / n respectively to obtain a plurality of n-1 interpolated deformation fields, wherein n is a natural number and k takes the value of each natural number in [1, n-1]; and a second processing subunit, configured to apply each interpolated deformation field to the three-dimensional image of the first phase respectively to obtain three-dimensional images of n-1 phases from the first phase to the second phase.

[0125] In some embodiments, the registration unit includes: a first determining subunit, configured to determine the projection matrix of the virtual transmitter based on the pose parameters of the virtual transmitter corresponding to the target two-dimensional image; a transformation subunit, configured to convert the first phase three-dimensional image into point cloud data of the target object while keeping the coordinate system unchanged; and a projection subunit, configured to project the point cloud data of the target object onto the real two-dimensional image using the projection matrix to obtain a registered image.

[0126] In some embodiments, the calculation unit includes repeatedly executing the following steps until the dissimilarity is less than a predetermined dissimilarity threshold, and then using the latest current deformation field as the deformation field of the target part from the first phase to the second phase: inputting the original images of the first phase three-dimensional image and the second phase three-dimensional image into an artificial intelligence neural network to obtain the current deformation field; applying the current deformation field to the first phase three-dimensional image to obtain the deformed first phase three-dimensional image; calculating the dissimilarity between the deformed first phase three-dimensional image and the original image of the first phase three-dimensional image; and adjusting the parameters of the artificial intelligence neural network if the dissimilarity is greater than or equal to the predetermined dissimilarity threshold.

[0127] In some embodiments, the second determining unit includes: a first calculation subunit, configured to calculate the similarity between each projected two-dimensional image in the projected two-dimensional image set and the real two-dimensional image respectively; and a first acquisition subunit, configured to take the projected two-dimensional image corresponding to the largest similarity value as the target two-dimensional image, and acquire the virtual emission source pose parameters corresponding to the target two-dimensional image.

[0128] In some embodiments, the apparatus further includes: a third determining unit, configured to determine the actual positional relationship between the transmitter of the imaging device used to capture a two-dimensional image of the patient site, the target patient site, and the imaging plate before adjusting the pose parameters of the virtual transmitter; and a fourth determining unit, configured to determine the virtual positional relationship between the virtual imaging plate, the first three-dimensional image, and the virtual transmitter based on the actual positional relationship; and to determine the pose parameter adjustment range of the virtual transmitter based on the virtual positional relationship, wherein the pose of the transmitter of the imaging device relative to the target patient site is located in the middle of the pose parameter adjustment range.

[0129] In some embodiments, the virtual projection unit includes: a search subunit, configured to use the pose of the imaging device's emission source relative to the patient's target site as the initial pose parameter of the virtual emission source, adjust the pose parameter of the virtual emission source starting from the initial pose parameter, and search for a target two-dimensional image; during the search for the target two-dimensional image, for each virtual emission source pose parameter determined, calculate the similarity between the projected two-dimensional image corresponding to the pose parameter and the real two-dimensional image, and if the similarity is greater than a predetermined similarity threshold, take the projected two-dimensional image corresponding to the largest similarity value as the target two-dimensional image, and obtain the virtual emission source pose parameter corresponding to the target two-dimensional image.

[0130] In some embodiments, the virtual projection unit further includes: a second acquisition subunit, configured to acquire the pose of the emission source relative to the patient target area recorded by the imaging device when capturing a real two-dimensional image; or, a third acquisition subunit, configured to acquire a first pose of the emission source and a second pose of the patient target area determined by the optical positioning system when capturing a real two-dimensional image; and a second calculation subunit, configured to determine the pose of the emission source of the imaging device relative to the patient target area based on the first pose and the second pose.

[0131] In some embodiments, the target site includes the lungs, the first phase three-dimensional image is an inspiratory three-dimensional image, and the second phase three-dimensional image is an expiratory three-dimensional image; or, the target site includes the heart or blood vessels, the first phase three-dimensional image is a diastolic three-dimensional image, and the second phase three-dimensional image is a systolic three-dimensional image.

[0132] The specific details of the above-mentioned devices can be understood from the relevant descriptions and effects in the method embodiments, and will not be repeated here.

[0133] The present invention also provides a surgical system, such as Figure 15 As shown, the surgical system includes the aforementioned image registration device, imaging equipment A, and surgical robot B.

[0134] Imaging device A is used to capture a true two-dimensional image of the target area of ​​the patient.

[0135] For example, imaging device A can capture real two-dimensional images at any time during or before the operation, depending on the needs of the surgery.

[0136] Imaging device A can be a Fluor imaging device.

[0137] Surgical robot B is used to perform surgical procedures based on image registration. For example, the surgeon can control surgical robot B to perform surgical procedures based on the displayed registration image, or the surgeon can obtain control instructions for surgical robot B by further processing the registration image, so that surgical robot B can automatically perform surgical procedures based on the registration image.

[0138] The image registration device described above can be implemented by an electronic device C.

[0139] In some embodiments, the surgical system further includes a ventilator D and a controller E for intervening in the patient's breathing process. Accordingly, the controller E can control the ventilator D to pause and control the imaging device A to capture a realistic two-dimensional image upon receiving a target operation command. The target operation command is used to determine the position of the target object in the realistic two-dimensional image.

[0140] For example, during a lung nodule biopsy, the surgeon needs to determine the relative position of the puncture needle to the lung nodule. This can be achieved by issuing a target operation command via interactive components such as buttons. In response to this command, controller E pauses the ventilator D and controls imaging device A to capture a real two-dimensional image. Image registration device or electronic device C determines a target two-dimensional image from the set of projected two-dimensional images that matches the real two-dimensional image, and determines the target pose parameters of the virtual emitter corresponding to the target two-dimensional image. Using these target pose parameters, the lung nodule in the inspiratory phase three-dimensional image is projected into the real two-dimensional image, resulting in a registered image. Based on the displayed registered image, the surgeon controls the surgical robot B to move the puncture needle.

[0141] The specific details of the above surgical system can be understood from the relevant descriptions and effects in the method embodiments, and will not be repeated here.

[0142] This invention also provides an electronic device, such as... Figure 16As shown, the electronic device may include a processor 1601 and a memory 1602, wherein the processor 1601 and the memory 1602 may be connected via a bus or other means. Figure 16 Taking the example of a connection between China and Israel via a bus.

[0143] The processor 1601 can be a central processing unit (CPU), etc.

[0144] The memory 1602, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the image registration method in this embodiment of the invention (e.g., the first acquisition unit 10, the calculation unit 20, the first determination unit 30, the virtual projection unit 40, the second acquisition unit 50, the second determination unit 60, and the registration unit 70). The processor 1601 executes various functional applications and data classification by running the non-transitory software programs, instructions, and modules stored in the memory 1602, thereby implementing the above-described image registration method.

[0145] The memory 1602 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the processor 1601, etc. Furthermore, the memory 1602 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 1602 may optionally include memory remotely located relative to the processor 1601, and these remote memories may be connected to the processor 1601 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0146] The one or more modules are stored in the memory 1602, and when executed by the processor 1601, the above-described image registration method is performed.

[0147] The specific details of the above-mentioned electronic device can be understood from the relevant descriptions and effects in the above embodiments, and will not be repeated here.

[0148] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. In particular, hardware + program embodiments are relatively simple in description because they are fundamentally similar to method embodiments; relevant parts can be referred to the descriptions in the method embodiments.

[0149] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0150] Those skilled in the art will also know that, besides implementing the controller using purely computer-readable program code, the same functions can be achieved by logically programming the method steps, making the controller function as logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers (PLCs), and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the devices within it used to implement various functions can also be considered structures within that hardware component. Alternatively, the devices used to implement various functions can be considered as both software modules implementing the method and structures within a hardware component.

[0151] The above description is merely an embodiment of the embodiments in this specification and is not intended to limit the embodiments of this specification. For those skilled in the art, various modifications and variations can be made to the embodiments of this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the embodiments of this specification should be included within the scope of the claims of the embodiments of this specification.

Claims

1. An image registration device, characterized in that, include: The first acquisition unit is used to acquire a first-phase three-dimensional image of the target site under expansion and a second-phase three-dimensional image of the target site under contraction, and to mark the target object in the first-phase three-dimensional image; The calculation unit is used to calculate the deformation field of the target part from the first phase to the second phase based on the first phase three-dimensional image and the second phase three-dimensional image; The first determining unit is used to determine a three-dimensional image of multiple phases from the first phase to the second phase based on the deformation field; The virtual projection unit is used to adjust the pose parameters of the virtual emission source for the acquired three-dimensional images of each phase, determine the projected two-dimensional images obtained by projecting the three-dimensional images of each phase under the position parameters of multiple emission sources, and obtain a set of projected two-dimensional images. The second acquisition unit is used to acquire a real two-dimensional image of the patient's target area; The second determining unit is used to determine the virtual emission source pose parameters of the target two-dimensional image that matches the real two-dimensional image in the projected two-dimensional image set; The registration unit is used to project the target object marked in the first phase three-dimensional image onto the real two-dimensional image using the virtual emission source pose parameters corresponding to the target two-dimensional image, so as to obtain a registration image.

2. The apparatus according to claim 1, characterized in that, The first determining unit includes: The first processing subunit is used to multiply the deformation field by k / n respectively to obtain a plurality of n-1 interpolated deformation fields, where n is a natural number and k takes the values ​​of various natural numbers in [1, n-1]. The second processing subunit is used to apply each interpolation deformation field to the three-dimensional image of the first phase to obtain three-dimensional images of n-1 phases from the first phase to the second phase.

3. The apparatus according to claim 1, characterized in that, The registration unit includes: The first determining subunit is used to determine the projection matrix of the virtual emission source based on the pose parameters of the virtual emission source corresponding to the target two-dimensional image; The transformation subunit is used to convert the first phase three-dimensional image into point cloud data of the target object while keeping the coordinate system unchanged; The projection subunit is used to project the point cloud data of the target object onto a real two-dimensional image using the projection matrix to obtain a registered image.

4. The apparatus according to claim 1, characterized in that, The calculation unit includes repeatedly executing the following steps until the dissimilarity is less than a predetermined dissimilarity threshold, and then using the latest current deformation field as the deformation field of the target part from the first phase to the second phase: The original images of the first phase 3D image and the second phase 3D image are input into the artificial intelligence neural network to obtain the current deformation field; The current deformation field is applied to the first phase three-dimensional image to obtain the deformed first phase three-dimensional image; Calculate the dissimilarity between the deformed three-dimensional image of the first phase and the original image of the first phase three-dimensional image; If the dissimilarity is greater than or equal to a predetermined dissimilarity threshold, the parameters of the artificial intelligence neural network are adjusted.

5. The apparatus according to claim 1, characterized in that, The second determining unit includes: The first calculation subunit is used to calculate the similarity between each projected two-dimensional image in the projected two-dimensional image set and the real two-dimensional image, respectively. The first acquisition subunit is used to take the projected two-dimensional image corresponding to the largest similarity value as the target two-dimensional image, and acquire the virtual emission source pose parameters corresponding to the target two-dimensional image.

6. The apparatus according to claim 1, characterized in that, Also includes: The third determining unit is used to determine the actual positional relationship between the imaging device's emission source, the patient's target area, and the imaging plate before adjusting the pose parameters of the virtual emission source. The fourth determining unit is used to determine the virtual positional relationship between the virtual imaging plate, the first phase three-dimensional image, and the virtual emission source based on the actual positional relationship. The pose parameter adjustment range of the virtual transmitter is determined based on the virtual positional relationship, and the pose of the transmitter of the imaging device relative to the target part of the patient is located in the middle of the pose parameter adjustment range.

7. The apparatus according to claim 6, characterized in that, The virtual projection unit includes: The search subunit is used to use the pose of the imaging device's emission source relative to the patient's target area as the initial pose parameter of the virtual emission source, and to adjust the pose parameter of the virtual emission source and search for the target two-dimensional image starting from the initial pose parameter. During the search for the target 2D image, for each virtual emitter whose pose parameters are determined, the similarity between the projected 2D image corresponding to the pose parameters and the real 2D image is calculated. If the similarity is greater than a predetermined similarity threshold, the projected 2D image with the highest similarity value is taken as the target 2D image, and the pose parameters of the virtual emitter corresponding to the target 2D image are obtained.

8. The apparatus according to claim 7, characterized in that, The virtual projection unit also includes: The second acquisition subunit is used to acquire the pose of the emission source relative to the patient's target area, as recorded by the imaging device when capturing a real two-dimensional image. or, The third acquisition subunit is used to acquire the first pose of the emission source and the second pose of the patient's target body determined by the optical positioning system when capturing real two-dimensional images. The second calculation subunit is used to determine the pose of the imaging device's emission source relative to the patient's target area based on the first pose and the second pose.

9. The apparatus according to claim 1, characterized in that, The target site includes the lungs, where the first phase three-dimensional image is an inspiratory three-dimensional image and the second phase three-dimensional image is an expiratory three-dimensional image; or, the target site includes the heart or blood vessels, where the first phase three-dimensional image is a diastolic three-dimensional image and the second phase three-dimensional image is a systolic three-dimensional image.

10. An image registration method, characterized in that, include: Acquire a first-phase three-dimensional image of the target site under expansion and a second-phase three-dimensional image of the target site under contraction, and mark the target object in the first-phase three-dimensional image; Calculate the deformation field of the target area from the first phase to the second phase based on the first phase three-dimensional image and the second phase three-dimensional image; A three-dimensional image of multiple phases from the first phase to the second phase is determined based on the deformation field; For the three-dimensional images of each phase that have been acquired, adjust the pose parameters of the virtual emission source, determine the projected two-dimensional images obtained by projecting the three-dimensional image of each phase under the position parameters of multiple emission sources, and obtain a set of projected two-dimensional images. Acquire realistic two-dimensional images of the target area on the patient; Determine the pose parameters of the virtual emitter of the target 2D image that matches the real 2D image in the set of projected 2D images; The target object marked in the first phase three-dimensional image is projected onto the real two-dimensional image using the virtual emission source pose parameters corresponding to the target two-dimensional image to obtain a registration image.

Images