Image-based registration methods and apparatus

Abstract

This application relates to an image-based registration method, apparatus, computer device, storage medium, and computer program product. The method includes: acquiring a 3D model constructed for a target object; acquiring a first 2D medical image and a second 2D medical image of the target object, wherein the pose of the 2D imaging device when acquiring the first 2D medical image is orthogonal to the pose when acquiring the second 2D medical image; acquiring a first transformation matrix between the digitally reconstructed 2D image coordinate system and the coordinate system of the 2D imaging device; based on the initial pose of the target object in the 3D model and the first transformation matrix, acquiring the planned pose of the target object in the coordinate system of the 2D imaging device; and transforming the planned pose to the robotic arm coordinate system to obtain the desired pose of the target object in the robotic arm coordinate system. This method can improve the accuracy of robotic arm operations.

Description

Technical Field

[0001] This application relates to the field of medical device technology, and in particular to an image-based registration method and apparatus. Background Technology

[0002] During the surgery, the coordinate space of the surgical robot must first be correlated with the coordinate space of the patient's actual pose. Then, guided by the navigation system, the surgical robot operates within the pre-planned surgical area. The guidance method for the surgical robot typically employs optical or magnetic positioning-based navigation after a target is implanted into the patient's bone tissue. However, implanting a target into the patient's bone tissue can cause additional damage to the bone, reducing surgical safety. Summary of the Invention

[0003] This application provides an image-based registration method and apparatus that can improve surgical safety.

[0004] In a first aspect, this application provides an image-based registration method applied to a registration system comprising a two-dimensional imaging device and a robotic arm, wherein the two-dimensional imaging device and the robotic arm are fixedly connected; the method includes:

[0005] Based on preoperative 3D medical images, a 3D model constructed for the target object is obtained;

[0006] Acquire first and second two-dimensional medical images of the target object. The orientation of the two-dimensional imaging device when acquiring the first two-dimensional medical image is orthogonal to the orientation when acquiring the second two-dimensional medical image.

[0007] Based on the imaging parameters of the digitally reconstructed two-dimensional image obtained from the digital image reconstruction of the three-dimensional model, as well as the imaging parameters of the first two-dimensional medical imaging and the second two-dimensional medical imaging respectively, a first transformation matrix between the coordinate system of the digitally reconstructed two-dimensional image and the coordinate system of the two-dimensional imaging device is obtained.

[0008] Based on the initial pose and first transformation matrix of the target object in the 3D model, the planned pose of the target object in the coordinate system of the 2D imaging device is obtained.

[0009] The planned pose is transformed into the robot arm coordinate system to obtain the desired pose of the target object in the robot arm coordinate system.

[0010] By constructing a 3D model of the target object and using first- and second-dimensional medical imaging of the target object, the initial pose of the target object can be transformed into the robotic arm coordinate system. This allows the robotic arm to be guided in real-time within the surgical area, and the guidance accuracy can be ensured by using intraoperative images for correction. Furthermore, the 2D imaging device is integrated with the robotic arm, enabling the real-time transformation of the initial pose to the robotic arm coordinate system, thus obtaining the real-time pose of the target object within the robotic arm coordinate system and improving guidance accuracy.

[0011] In one embodiment, based on preoperative 3D medical images, a 3D model constructed for the target object is obtained, including:

[0012] The three-dimensional medical images are sequentially processed by interpolation, noise reduction, and contrast enhancement. Based on the processed three-dimensional medical images, three-dimensional reconstruction is performed to obtain a three-dimensional model constructed for the target object.

[0013] By denoising the interpolated 3D medical images, the impact of noise on the 3D reconstruction results can be reduced; by enhancing the contrast of the denoised 3D medical images, the difference between the region of interest and other regions in the 3D medical images can be increased.

[0014] In one embodiment, based on the imaging parameters of the digitally reconstructed two-dimensional image obtained from digital image reconstruction of the three-dimensional model, and the imaging parameters of the first and second two-dimensional medical imaging respectively, a first transformation matrix between the coordinate system of the digitally reconstructed two-dimensional image and the coordinate system of the two-dimensional imaging device is obtained, including:

[0015] The digitally reconstructed two-dimensional image is registered with the first two-dimensional medical image and the second two-dimensional medical image respectively to obtain the digitally reconstructed two-dimensional image correspondingly registered with the first two-dimensional medical image and the second two-dimensional medical image respectively.

[0016] The first transformation matrix is ​​determined based on the imaging parameters of the first two-dimensional medical imaging and the second two-dimensional medical imaging, as well as the digital reconstruction imaging parameters of the digitally reconstructed two-dimensional images corresponding to the registration of the first two-dimensional medical imaging and the digitally reconstructed two-dimensional images corresponding to the registration of the second two-dimensional medical imaging.

[0017] By acquiring first and second two-dimensional medical images during surgery and registering them with digitally reconstructed two-dimensional images, and determining the first transformation matrix based on the registration results, the real-time pose of the target object in the robotic arm coordinate system can be obtained, thereby improving the accuracy of robotic arm operation.

[0018] In one embodiment, the two-dimensional imaging device includes a C-arm and an X-ray tube; when a first two-dimensional medical image is obtained through imaging via the X-ray tube, the C-arm is in a first posture; when a second two-dimensional medical image is obtained through imaging via the X-ray tube, the C-arm is in a second posture.

[0019] By using first two-dimensional medical imaging and second two-dimensional medical imaging, a digitally reconstructed two-dimensional image registered with the first two-dimensional medical imaging and the second two-dimensional medical imaging can be determined from multiple digitally reconstructed two-dimensional images, thereby determining the first transformation matrix between the coordinate system of the digitally reconstructed two-dimensional image and the coordinate system of the two-dimensional imaging device.

[0020] In one embodiment, the digitally reconstructed two-dimensional image is registered with a first two-dimensional medical image and a second two-dimensional medical image, respectively, to obtain digitally reconstructed two-dimensional images that are respectively registered with the first two-dimensional medical image and the second two-dimensional medical image, including:

[0021] Digital image reconstruction is performed on the 3D model to obtain at least one digitally reconstructed 2D image;

[0022] The first two-dimensional medical image and the second two-dimensional medical image are registered with the digitally reconstructed two-dimensional image respectively. If the first two-dimensional medical image or the second two-dimensional medical image does not determine the corresponding registered digitally reconstructed two-dimensional image, the digital image reconstruction parameters are updated and the three-dimensional model is digitally reconstructed. Multiple digitally reconstructed two-dimensional images are obtained again and re-registered. The above update, reconstruction and registration process is repeated until the first two-dimensional medical image and the second two-dimensional medical image both determine the corresponding registered digitally reconstructed two-dimensional images, and the corresponding registered digitally reconstructed two-dimensional images of the first two-dimensional medical image and the second two-dimensional medical image are obtained.

[0023] By using an image registration algorithm to determine the image registered with the first two-dimensional medical imaging and the second two-dimensional medical imaging, a first transformation matrix between the preoperative image coordinate system and the coordinate system of the two-dimensional imaging device can be determined. This allows the real-time pose of the target object in the preoperative image coordinate system to be converted into the real-time pose in the robotic arm coordinate system, avoiding the need to install additional targets on the target object, thereby improving the safety of the target object and shortening the recovery time of the target object.

[0024] In one embodiment, digital image reconstruction is performed on the three-dimensional model to obtain at least one digitally reconstructed two-dimensional image, including:

[0025] Obtain digital reconstruction imaging parameters, including the location of the simulated light source and the projection angle of the simulated light source onto the 3D model;

[0026] Based on the projection angle and location, light rays are emitted from a simulated light source onto the 3D model, and the coordinates of the intersection point between the light rays emitted from the simulated light source and the 3D model are calculated.

[0027] The grayscale values ​​of the intersection points of each ray are obtained by trilinear interpolation, and the grayscale values ​​of the intersection points of each ray are accumulated.

[0028] The accumulated gray values ​​of each ray are assigned to the intersection point between each ray and the digital imaging plane to obtain the digital reconstructed two-dimensional image corresponding to the digital reconstruction imaging parameters.

[0029] By performing digital image reconstruction on a three-dimensional model, multiple digitally reconstructed two-dimensional images can be obtained, which can then be registered with the first and second two-dimensional medical images during the operation.

[0030] Secondly, this application also provides an image-based registration device. Applied to a registration system comprising a two-dimensional imaging device and a robotic arm, the two-dimensional imaging device and the robotic arm being fixedly connected; the device includes:

[0031] The first acquisition module is used to acquire a three-dimensional model of the target object based on the three-dimensional medical images taken before surgery.

[0032] The second acquisition module is used to acquire the first two-dimensional medical image and the second two-dimensional medical image of the target object. The posture of the two-dimensional imaging device when acquiring the first two-dimensional medical image is orthogonal to the posture when acquiring the second two-dimensional medical image.

[0033] The reconstruction module is used to obtain the first transformation matrix between the coordinate system of the digitally reconstructed two-dimensional image and the coordinate system of the two-dimensional imaging device based on the imaging parameters of the digitally reconstructed two-dimensional image obtained by digital image reconstruction of the three-dimensional model, as well as the imaging parameters of the first two-dimensional medical imaging and the second two-dimensional medical imaging respectively.

[0034] The third acquisition module is used to acquire the planned pose of the target object in the coordinate system of the two-dimensional imaging device based on the initial pose of the target object in the three-dimensional model and the first transformation matrix.

[0035] The configuration module is used to convert the planned pose to the robot arm coordinate system to obtain the desired pose of the target object in the robot arm coordinate system.

[0036] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0037] Based on preoperative 3D medical images, a 3D model constructed for the target object is obtained;

[0038] Acquire first and second two-dimensional medical images of the target object. The orientation of the two-dimensional imaging device when acquiring the first two-dimensional medical image is orthogonal to the orientation when acquiring the second two-dimensional medical image.

[0039] Based on the imaging parameters of the digitally reconstructed two-dimensional image obtained from the digital image reconstruction of the three-dimensional model, as well as the imaging parameters of the first two-dimensional medical imaging and the second two-dimensional medical imaging respectively, a first transformation matrix between the coordinate system of the digitally reconstructed two-dimensional image and the coordinate system of the two-dimensional imaging device is obtained.

[0040] Based on the initial pose and first transformation matrix of the target object in the 3D model, the planned pose of the target object in the coordinate system of the 2D imaging device is obtained.

[0041] The planned pose is transformed into the robot arm coordinate system to obtain the desired pose of the target object in the robot arm coordinate system.

[0042] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0043] Based on preoperative 3D medical images, a 3D model constructed for the target object is obtained;

[0044] Acquire first and second two-dimensional medical images of the target object. The orientation of the two-dimensional imaging device when acquiring the first two-dimensional medical image is orthogonal to the orientation when acquiring the second two-dimensional medical image.

[0045] Based on the imaging parameters of the digitally reconstructed two-dimensional image obtained from the digital image reconstruction of the three-dimensional model, as well as the imaging parameters of the first two-dimensional medical imaging and the second two-dimensional medical imaging respectively, a first transformation matrix between the coordinate system of the digitally reconstructed two-dimensional image and the coordinate system of the two-dimensional imaging device is obtained.

[0046] Based on the initial pose and first transformation matrix of the target object in the 3D model, the planned pose of the target object in the coordinate system of the 2D imaging device is obtained.

[0047] The planned pose is transformed into the robot arm coordinate system to obtain the desired pose of the target object in the robot arm coordinate system.

[0048] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0049] Based on preoperative 3D medical images, a 3D model constructed for the target object is obtained;

[0050] Acquire first and second two-dimensional medical images of the target object. The orientation of the two-dimensional imaging device when acquiring the first two-dimensional medical image is orthogonal to the orientation when acquiring the second two-dimensional medical image.

[0051] Based on the imaging parameters of the digitally reconstructed two-dimensional image obtained from the digital image reconstruction of the three-dimensional model, as well as the imaging parameters of the first two-dimensional medical imaging and the second two-dimensional medical imaging respectively, a first transformation matrix between the coordinate system of the digitally reconstructed two-dimensional image and the coordinate system of the two-dimensional imaging device is obtained.

[0052] Based on the initial pose and first transformation matrix of the target object in the 3D model, the planned pose of the target object in the coordinate system of the 2D imaging device is obtained.

[0053] The planned pose is transformed into the robot arm coordinate system to obtain the desired pose of the target object in the robot arm coordinate system.

[0054] The aforementioned image-based registration method, apparatus, computer equipment, storage medium, and computer program product acquire a 3D model of the target object based on preoperative 3D medical images; acquire first and second 2D medical images of the target object, with the pose of the 2D imaging device when acquiring the first 2D medical image being orthogonal to the pose when acquiring the second 2D medical image; based on the imaging parameters of the digitally reconstructed 2D image obtained from digital image reconstruction of the 3D model, and the respective imaging parameters of the first and second 2D medical images, obtain a first transformation matrix between the coordinate system of the digitally reconstructed 2D image and the coordinate system of the 2D imaging device; based on the initial pose of the target object in the 3D model and the first transformation matrix, obtain the planned pose of the target object in the coordinate system of the 2D imaging device; transform the planned pose to the robotic arm coordinate system to obtain the desired pose of the target object in the robotic arm coordinate system. This method eliminates the need to implant a target at the target object, thus ensuring the safety of the target object and shortening the recovery time; furthermore, by acquiring intraoperative images for registration in real time, the accuracy of robotic arm operation can be improved. Attached Figure Description

[0055] Figure 1 This is a flowchart illustrating an image-based registration method in one embodiment;

[0056] Figure 2 This is a schematic diagram illustrating the acquisition of a digitally reconstructed two-dimensional image in one embodiment;

[0057] Figure 3 This is a schematic diagram of three-dimensional reconstruction in one embodiment;

[0058] Figure 4 This is a schematic diagram of the preprocessing flow of CT images in one embodiment;

[0059] Figure 5 This is a schematic diagram of preoperative planning in one embodiment;

[0060] Figure 6 This is a registration diagram in one embodiment;

[0061] Figure 7 This is a schematic diagram of a scene where a first X-ray image is acquired in one embodiment;

[0062] Figure 8 This is a schematic diagram of a scenario for acquiring a second X-ray image in one embodiment;

[0063] Figure 9 This is a schematic diagram of acquiring X-ray images of the knee joint in one embodiment;

[0064] Figure 10 This is a schematic diagram of the image registration process in one embodiment;

[0065] Figure 11 This is a schematic diagram of the process of obtaining a DRR image in one embodiment;

[0066] Figure 12 This is a schematic diagram of the process of digital image reconstruction of a 3D model in one embodiment;

[0067] Figure 13 This is a schematic diagram of the surgical system in one embodiment;

[0068] Figure 14 Here is a system structure block diagram from one embodiment;

[0069] Figure 15 This is a schematic diagram of the coordinate transformation process in one embodiment;

[0070] Figure 16 This is a flowchart of an image-based registration method in one embodiment;

[0071] Figure 17 This is a structural block diagram of an image-based registration device in one embodiment;

[0072] Figure 18 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0073] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0074] This application provides an image-based registration method that eliminates the need for additional targets on the target object, thereby improving the safety of the target object and shortening its recovery time. Furthermore, by using real-time surgical area images acquired during surgery, image registration and coordinate transformation can increase the accuracy of robotic arm operations. In addition, this application does not require additional calibration devices, which can reduce costs and simplify preoperative preparation steps.

[0075] like Figure 1 As shown, an image-based registration method is provided, applied to a registration system comprising a two-dimensional imaging device and a robotic arm, the two-dimensional imaging device and the robotic arm being fixedly connected. In this embodiment, the method includes the following steps:

[0076] 101. Based on preoperative 3D medical images, obtain a 3D model constructed for the target object;

[0077] 102. Acquire the first two-dimensional medical image and the second two-dimensional medical image of the target object. The posture of the two-dimensional imaging device when acquiring the first two-dimensional medical image is orthogonal to the posture when acquiring the second two-dimensional medical image.

[0078] 103. Based on the imaging parameters of the digitally reconstructed two-dimensional image obtained by digital image reconstruction of the three-dimensional model, and the imaging parameters of the first two-dimensional medical imaging and the second two-dimensional medical imaging respectively, obtain the first transformation matrix between the coordinate system of the digitally reconstructed two-dimensional image and the coordinate system of the two-dimensional imaging device.

[0079] 104. Based on the initial pose and first transformation matrix of the target object in the 3D model, obtain the planned pose of the target object in the coordinate system of the 2D imaging device;

[0080] 105. Transform the planned pose into the robot arm coordinate system to obtain the desired pose of the target object in the robot arm coordinate system.

[0081] In this context, 3D medical imaging can refer to CT images; the target object refers to the tissue or organ at the site of disease, as determined by a doctor's diagnosis, such as the knee joint. The initial pose of the target object can be a single pose or a set of ordered or unordered poses. For example, the initial pose of the target object can be a set of all poses within the surgical area determined by the doctor, or a pose corresponding to the surgical procedure steps determined by the doctor. The initial pose of the target object is obtained within the coordinate system of the digitally reconstructed 2D image (or the coordinate system of the 3D model). 2D imaging equipment refers to the device used to capture images of the target object during surgery to obtain 2D medical images.

[0082] There is a fixed geometric positional relationship between the 2D imaging device and the robotic arm. The second transformation matrix between the coordinate system of the 2D imaging device and the coordinate system of the robotic arm is determined. The transformation matrix between the coordinate system of the digitally reconstructed 2D image and the coordinate system of the 2D imaging device is used as the first transformation matrix, and the transformation matrix between the coordinate system of the 2D imaging device and the coordinate system of the robotic arm is used as the second transformation matrix. Through the first transformation matrix, the initial pose of the target object in the coordinate system of the digitally reconstructed 2D image can be transformed to the coordinate system of the 2D imaging device, thus obtaining the planned pose of the target object. Through the second transformation matrix, the planned pose of the target object in the coordinate system of the 2D imaging device can be transformed to the coordinate system of the robotic arm, thus obtaining the desired pose of the target object in the coordinate system of the robotic arm.

[0083] The process of determining the second transformation matrix includes: using P c Let P represent the coordinates of any point in the coordinate system of the two-dimensional imaging device. r Point P represents the midpoint of the coordinate system of the two-dimensional imaging device. c In the robotic arm coordinate system, (x1, y1, z1) represent the offsets of the origin of the 2D imaging device coordinate system relative to the origin of the robotic arm coordinate system along the x, y, and z axes, respectively, when the robotic arm coordinate system is used as the reference coordinate system. When setting up the 2D imaging device coordinate system and the robotic arm coordinate system, the x, y, and z axes of the 2D imaging device coordinate system are set to the same coordinate axis directions as the corresponding x, y, and z axes of the robotic arm coordinate system. Therefore, when performing coordinate system transformations, only the translation transformation between the 2D imaging device coordinate system and the robotic arm coordinate system needs to be considered. Thus, the transformation relationship between the 2D imaging device coordinate system and the robotic arm coordinate system is: P r =D*P c Where D is the translation operator, expressed as: .

[0084] The orthogonality between the posture of a two-dimensional imaging device when it captures the first two-dimensional medical image and the posture when it captures the second two-dimensional medical image means that the posture of the two-dimensional imaging device when capturing the first two-dimensional medical image is orthogonal to the posture of the two-dimensional imaging device when capturing the second two-dimensional medical image. For example, the two-dimensional imaging device captures the first two-dimensional medical image from the front of the target object, and the two-dimensional imaging device captures the first two-dimensional medical image from the side of the target object.

[0085] If the digitally reconstructed 2D image is obtained through an X-ray imaging device, then the digitally reconstructed 2D image obtained by Digitally Reconstructed Radiograph (DRR) processing is also called a digitally reconstructed radiographic image. In this case, the imaging parameters of the digitally reconstructed 2D image include the projection distance and projection angle, where the projection distance refers to the distance between the X-ray source and the 3D model. An example is shown below, illustrating the process of obtaining a digitally reconstructed 2D image from a 3D model. Figure 2 As shown, based on the three-dimensional model, the imaging principle of the simulated X-ray system is simulated. By changing the distance and projection angle between the simulated X-ray source and the three-dimensional model, a digitally reconstructed two-dimensional image is generated on the simulated DRR imaging plane.

[0086] Specifically, 3D medical image data acquired preoperatively from the target object is used for 3D reconstruction to obtain a 3D model of the target object. Digital image reconstruction is then performed on the 3D model to obtain multiple digitally reconstructed 2D images and corresponding image parameters for each image. During the operation, 2D medical images of the target object in two poses (i.e., first 2D medical imaging and second 2D medical imaging) are acquired in real time using a 2D imaging device, and the corresponding imaging parameters for each are obtained. Based on the imaging parameters of the digitally reconstructed 2D images obtained from the digital image reconstruction and the corresponding imaging parameters of the first and second 2D medical images, a first transformation matrix is ​​obtained between the coordinate system of the digitally reconstructed 2D images and the coordinate system of the 2D imaging device. This first transformation matrix is ​​used to transform the initial pose of the target object to the coordinate system of the 2D imaging device during the operation, obtaining the planned pose of the target object. A second transformation matrix is ​​then used to further transform the planned pose from the coordinate system of the 2D imaging device to the coordinate system of the robotic arm.

[0087] The method provided in this invention, through a three-dimensional model constructed for the target object and first and second two-dimensional medical imaging of the target object, can transform the initial pose of the target object into the robotic arm coordinate system. This allows for real-time guidance of the robotic arm in the surgical area, and the guidance accuracy of the robotic arm can be ensured by using intraoperative images for correction. Furthermore, the two-dimensional imaging device and the robotic arm are integrated, enabling the real-time transformation of the initial pose to the robotic arm coordinate system, obtaining the real-time pose of the target object within the robotic arm coordinate system, thereby improving guidance accuracy.

[0088] like Figure 3 and Figure 4 As shown, in one embodiment, obtaining a 3D model of the target object based on preoperative 3D medical images includes:

[0089] The three-dimensional medical images are sequentially processed by interpolation, noise reduction, and contrast enhancement. Based on the processed three-dimensional medical images, three-dimensional reconstruction is performed to obtain a three-dimensional model constructed for the target object.

[0090] Interpolation processing refers to interpolating three-dimensional medical images based on parameters such as field of view, reconstruction matrix, and slice thickness to ensure isotropy.

[0091] Specifically, the three-dimensional medical images obtained from the preoperative scan of the target object are subjected to a series of preprocessing processes, such as interpolation, noise reduction, and contrast enhancement, to obtain preprocessed image data. Based on the preprocessed image data, a three-dimensional reconstruction is performed to obtain a three-dimensional model of the target object. Subsequent preoperative planning will be based on the three-dimensional model, such as determining the surgical area based on the three-dimensional model.

[0092] The method provided in this embodiment of the invention can reduce the impact of noise on the three-dimensional reconstruction results by performing noise reduction processing on the interpolated three-dimensional medical image; and can increase the difference between the target object and other parts in the three-dimensional medical image by performing contrast enhancement processing on the noise-reduced three-dimensional medical image.

[0093] In conjunction with the above embodiments, in one embodiment, based on the imaging parameters of the digitally reconstructed two-dimensional image obtained from digital image reconstruction of the three-dimensional model, and the imaging parameters of the first two-dimensional medical imaging and the second two-dimensional medical imaging respectively, a first transformation matrix between the coordinate system of the digitally reconstructed two-dimensional image and the coordinate system of the two-dimensional imaging device is obtained, including:

[0094] The digitally reconstructed two-dimensional image is registered with the first two-dimensional medical image and the second two-dimensional medical image respectively to obtain the digitally reconstructed two-dimensional image correspondingly registered with the first two-dimensional medical image and the second two-dimensional medical image respectively.

[0095] The first transformation matrix is ​​determined based on the imaging parameters of the first two-dimensional medical imaging and the second two-dimensional medical imaging, as well as the digital reconstruction imaging parameters of the digitally reconstructed two-dimensional images corresponding to the registration of the first two-dimensional medical imaging and the digitally reconstructed two-dimensional images corresponding to the registration of the second two-dimensional medical imaging.

[0096] The imaging parameters of both the first and second 2D medical images include imaging distance and imaging angle. Before registering the digitally reconstructed 2D image with the first and second 2D medical images, image normalization processing is required. Registration refers to comparing the similarity of the images. If the similarity between two images is greater than a preset threshold, the two images are considered successfully registered; if the similarity is not greater than the preset threshold, the two images are considered to have failed to register.

[0097] In addition, the similarity between two images can be determined by comparing their grayscale or contour information. For example, grayscale histogram matching can be performed on the two images. If the difference between the grayscale histogram of the first or second 2D medical image during surgery and the digitally reconstructed 2D image is less than a preset difference threshold, then the similarity between the two images can be considered to be greater than the preset threshold, and the two images are successfully registered.

[0098] In one example, taking osteotomy surgery as an example, the location of the osteotomy is determined through preoperative planning, such as... Figure 5 The area is marked with a line in the middle. During the operation, the target pose of the position to be cut off is transformed into the coordinate system of the robotic arm through image registration and pose transformation. The desired pose of the position to be cut off in the coordinate system of the robotic arm is obtained. The robotic arm of the surgical robot independently or assists the doctor in completing the osteotomy operation based on the desired pose in the coordinate system of the robotic arm.

[0099] Figure 6 This is a registration diagram. Figure 6 The left image shows the pose of the target object during a CT scan, while the right image shows the pose of the target object during intraoperative two-dimensional medical imaging using a two-dimensional imaging device.

[0100] Specifically, multiple digitally reconstructed 2D images can be obtained from digital image reconstruction of the 3D model. The similarity between the first 2D medical image and each digitally reconstructed 2D image is determined, and the digitally reconstructed 2D image with the highest similarity (greater than a preset similarity) is used as the registration image for the first 2D medical image. Similarly, the similarity between the second 2D medical image and each digitally reconstructed 2D image is determined, and the digitally reconstructed 2D image with the highest similarity (greater than a preset similarity) is used as the registration image for the second 2D medical image. Based on the imaging distance and angle between the first and second 2D medical images, and the imaging distance and projection angle between the registration images of the first and second 2D medical images, a first transformation matrix is ​​determined between the coordinate system of the digitally reconstructed 2D image and the coordinate system of the 2D imaging device (i.e., the coordinate system of the 2D imaging device).

[0101] The method provided in this embodiment of the invention obtains the real-time pose of the target object in the robotic arm coordinate system by registering the first two-dimensional medical image and the second two-dimensional medical image acquired during the operation with the digitally reconstructed two-dimensional image and determining the first transformation matrix based on the registration result, thereby improving the accuracy of the robotic arm operation.

[0102] In conjunction with the above embodiments, in one embodiment, the two-dimensional imaging device includes a C-arm and an X-ray tube; when a first two-dimensional medical image is obtained through imaging via the X-ray tube, the C-arm is in a first posture; when a second two-dimensional medical image is obtained through imaging via the X-ray tube, the C-arm is in a second posture.

[0103] The first and second postures are orthogonal. This means that the angle between the placement angle of the C-arm corresponding to the first posture and the placement angle of the C-arm corresponding to the second posture is 90 degrees. For example, if the placement angle of the C-arm corresponding to the first posture is parallel to the horizontal plane, then the placement angle of the C-arm corresponding to the second posture should be perpendicular to the horizontal plane, and the first and second postures are orthogonal.

[0104] In one example, the scene image for acquiring the first two-dimensional medical image is as follows: Figure 7 As shown, Figure 7 This is a schematic diagram for capturing an image of the front of a target object. Figure 7 The system consists of four parts: a C-arm X-ray machine 1, a C-arm X-ray tube 2, a C-arm imaging panel 3, and a target object 4. With the C-arm X-ray tube 2 positioned at 0°, the target object 4 is placed in a suitable position and fixed. X-ray images of the target object are acquired, serving as the first two-dimensional medical imaging. The scene for acquiring the second two-dimensional medical imaging is shown in the image below. Figure 8 As shown, Figure 8 This is a schematic diagram for capturing an image of the side view of a target object. Figure 8 It consists of four parts: a C-arm X-ray machine 1, a C-arm X-ray tube 2, a C-arm imaging panel 3, and a target object 4. At this time, the C-arm X-ray tube is placed at 90°, the target object is placed in a suitable position and fixed, and the X-ray image of the target object is acquired as a second two-dimensional medical imaging.

[0105] In one example, taking the knee joint as an example, the X-ray tube of the C-arm in the two-dimensional imaging device is placed at 0° and 90° to acquire X-ray images of the target object, as shown in the following diagram. Figure 9 As shown; X-ray images acquired at these two angles were registered with digitally reconstructed two-dimensional images.

[0106] The method provided in this embodiment of the invention can determine a digitally reconstructed two-dimensional image registered with the first two-dimensional medical image and the second two-dimensional medical image from multiple digitally reconstructed two-dimensional images through the first two-dimensional medical imaging and the second two-dimensional medical imaging, thereby determining a first transformation matrix between the coordinate system of the digitally reconstructed two-dimensional image and the coordinate system of the two-dimensional imaging device.

[0107] In conjunction with the above embodiments, in one embodiment, the digitally reconstructed two-dimensional image is registered with a first two-dimensional medical image and a second two-dimensional medical image respectively to obtain digitally reconstructed two-dimensional images correspondingly registered with the first two-dimensional medical image and the second two-dimensional medical image, including:

[0108] Digital image reconstruction is performed on the 3D model to obtain at least one digitally reconstructed 2D image;

[0109] The first two-dimensional medical image and the second two-dimensional medical image are registered with the digitally reconstructed two-dimensional image respectively. If the first two-dimensional medical image or the second two-dimensional medical image does not determine the corresponding registered digitally reconstructed two-dimensional image, the digital image reconstruction parameters are updated and the three-dimensional model is digitally reconstructed. Multiple digitally reconstructed two-dimensional images are obtained again and re-registered. The above update, reconstruction and registration process is repeated until the first two-dimensional medical image and the second two-dimensional medical image both determine the corresponding registered digitally reconstructed two-dimensional images, and the corresponding registered digitally reconstructed two-dimensional images of the first two-dimensional medical image and the second two-dimensional medical image are obtained.

[0110] Figure 10 This is a schematic diagram illustrating the process of registering digitally reconstructed 2D images obtained from digital image reconstruction of a 3D model with first-dimensional and second-dimensional medical images. Here, DRR parameters refer to the imaging parameters of the digitally reconstructed 2D images, including projection distance and projection angle; DRR image refers to the digitally reconstructed 2D image; and X-ray image refers to the first-dimensional and second-dimensional medical images obtained by 2D imaging equipment, including X-ray imaging.

[0111] Figure 11 The flowchart for acquiring DRR images is as follows: acquiring preoperative CT images and inputting them into the processing system; performing 3D reconstruction using the preoperative CT images to obtain a 3D model of the target object; performing preoperative planning based on the 3D model of the target object to determine the surgical area and record the pose within the planned surgical area; performing DRR processing based on the 3D model of the target object, obtaining several virtual DRR images by changing parameters such as the projection angle or projection distance of the DRR, and recording the imaging parameters of the DRR images.

[0112] Specifically, at least one preoperative 3D medical image data is used for 3D reconstruction to obtain a 3D model of the target object; DRR processing is performed based on the 3D model of the target object, and multiple DRR images are obtained by changing the projection distance and projection angle of the DRR, and the DRR parameters corresponding to each DRR image are recorded; during the operation, a 2D imaging device acquires the first and second 2D medical images of the target object; the DRR images, the first and second 2D medical images are normalized and similarity is compared; if the DRR image and the 2D medical image meet the similarity judgment condition, the registration is successful, and the corresponding DRR parameters are output for subsequent coordinate transformation; if the first or second 2D medical image does not determine the corresponding registered digital reconstructed radiographic image, the DRR parameters are updated, a new DRR image is acquired, and registration is performed on the new DRR image with the first and second 2D medical images respectively, until the first and second 2D medical images both determine the corresponding registered digital reconstructed radiographic images.

[0113] The method provided in this invention determines the image registered with the first two-dimensional medical imaging and the second two-dimensional medical imaging through an image registration algorithm. It can determine the first transformation matrix between the preoperative image coordinate system and the coordinate system of the two-dimensional imaging device, thereby realizing the conversion of the real-time pose of the target object in the preoperative image coordinate system into the real-time pose in the robotic arm coordinate system. This avoids the installation of additional targets at the target object, thereby improving the safety of the target object and shortening the recovery time of the target object.

[0114] In conjunction with the above embodiments, in one embodiment, digital image reconstruction is performed on a three-dimensional model to obtain at least one digitally reconstructed two-dimensional image, including:

[0115] Obtain digital reconstruction imaging parameters, including the location of the simulated light source and the projection angle of the simulated light source onto the 3D model;

[0116] Based on the projection angle and location, light rays are emitted from a simulated light source onto the 3D model, and the coordinates of the intersection point between the light rays emitted from the simulated light source and the 3D model are calculated.

[0117] The grayscale values ​​of the intersection points of each ray are obtained by trilinear interpolation, and the grayscale values ​​of the intersection points of each ray are accumulated.

[0118] The accumulated gray values ​​of each ray are assigned to the intersection point between each ray and the digital imaging plane to obtain the digital reconstructed two-dimensional image corresponding to the digital reconstruction imaging parameters.

[0119] In this embodiment, the digital image reconstruction algorithm used in the digital image reconstruction of the 3D model is the ray casting algorithm, such as... Figure 12 The diagram illustrates the process of digital image reconstruction of a 3D model, which includes setting the light source, ray tracing, linear interpolation, grayscale accumulation, and image mapping. Specifically, first, by selecting the location of the simulated light source, several rays are emitted from it, each ray corresponding to a pixel in the 2D image plane of the digital reconstruction. The projection angle and distance of the simulated light source relative to the 3D model are determined. Then, the coordinates of the intersection points of the rays emitted by the simulated light source with the 3D model are calculated, and the grayscale values ​​of these intersection points are recorded after trilinear interpolation. The grayscale values ​​of each ray passing through the 3D model and its various intersection points are accumulated, and the accumulated grayscale values ​​of each ray's intersection with the 3D model are assigned to the intersection points of the ray and the DRR imaging plane, thus obtaining the DRR image.

[0120] The method provided in this invention can obtain multiple digitally reconstructed two-dimensional images by performing digital image reconstruction on a three-dimensional model, thereby enabling the digitally reconstructed two-dimensional images to be registered with the first two-dimensional medical imaging and the second two-dimensional medical imaging during the operation.

[0121] Figure 13 This is a schematic diagram of a surgical system in an application scenario, including a robotic arm 1, a connection device between the robotic arm and a C-arm 2, a C-arm X-ray machine 3, a C-arm X-ray tube 4, a C-arm imaging panel 5, and a target object 6.

[0122] Figure 14 The diagram below shows the system structure in one embodiment. Specifically, through image registration, the target pose in the coordinate system of the CT image (i.e., the digital reconstruction two-dimensional image coordinate system) is first transformed to the coordinate system of the X-ray system (i.e., the two-dimensional imaging device coordinate system) to obtain the planned pose. Then, the planned pose is transformed to the robotic arm coordinate system.

[0123] Figure 15 This is a schematic diagram of the coordinate transformation process in one embodiment, including: acquiring preoperative CT images and inputting them into the system, followed by 3D reconstruction, at which point the CT image coordinates (or pose) can be obtained; based on the registration result of the intraoperative image and the preoperative image, the preoperative CT image coordinates (or pose) are converted into intraoperative X-ray image coordinates (or pose) in the coordinate system of the 2D imaging device; the coordinate system of the 2D imaging device and the C-arm coordinate system belong to the same coordinate system. Since the C-arm and the robotic arm are directly connected, the second transformation matrix between the C-arm coordinate system and the robotic arm coordinate system is known, so the X-ray system coordinates (or pose) can be directly converted into coordinates in the robotic arm coordinate system. Ultimately, the CT image coordinates (or pose) are transformed into the robotic arm coordinate system.

[0124] In one embodiment, such as Figure 16As shown, an image-based registration method includes:

[0125] 1601. Input the CT image containing the target object acquired before surgery;

[0126] 1602. Use the input CT images to perform 3D reconstruction to obtain a 3D model of the target object;

[0127] 1603. Based on the 3D reconstruction results, perform preoperative planning, determine the surgical area, and save the preoperative planning results;

[0128] 1604. Perform multi-parameter DRR reconstruction on the 3D reconstruction results to obtain several DRR images and record the corresponding DRR parameters;

[0129] 1605. No target needs to be implanted in the target object during the operation. X-ray images of the target object at 0° and 90° are acquired using a C-arm X-ray imaging device.

[0130] 1606. Register the intraoperative X-ray images with the preoperative DRR images and save the projection parameters of the registered DRR images;

[0131] 1607. Using the registration results, the initial coordinates (or pose) of the target object in the preoperative image coordinate system are transformed to the intraoperative X-ray image coordinate system to obtain the planned coordinates (or pose) of the target object. The planned coordinates (or pose) are then further transformed to the robotic arm coordinate system via the C-arm coordinate system to obtain the desired coordinates (or pose) of the target object.

[0132] 1608. Guide the robotic arm operation using the desired coordinates (or pose).

[0133] The method provided in this invention enables pose transformation within the surgical area through image registration, eliminating the need for additional targets at the target site. This improves the safety of the target, reduces the burden on surgeons, and shortens recovery time. Furthermore, using real-time intraoperative X-ray images allows for real-time robot guidance based on the actual information of the target, improving the accuracy of surgical guidance. Additionally, registering intraoperative 2D X-ray images with preoperative 3D images reduces the radiation dose received by the target during surgery. Finally, intraoperative image acquisition, registration, coordinate or pose transformation, and surgical guidance are all completed within the registration system, reducing time errors and improving the robotic arm's operational precision.

[0134] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0135] Based on the same inventive concept, this application also provides an image-based registration apparatus for implementing the image-based registration method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more image-based registration apparatus embodiments provided below can be found in the limitations of the image-based registration method described above, and will not be repeated here.

[0136] In one embodiment, such as Figure 17 As shown, an image-based registration device is provided, applied to a registration system including a two-dimensional imaging device and a robotic arm, wherein the two-dimensional imaging device and the robotic arm are fixedly connected; the device includes: a first acquisition module 1701, a second acquisition module 1702, a reconstruction module 1703, a third acquisition module 1704, and a configuration module 1705, wherein:

[0137] The first acquisition module 1701 is used to acquire a three-dimensional model constructed for the target object based on the three-dimensional medical images taken before surgery.

[0138] The second acquisition module 1702 is used to acquire the first two-dimensional medical image and the second two-dimensional medical image of the target object. The posture of the two-dimensional imaging device when acquiring the first two-dimensional medical image is orthogonal to the posture when acquiring the second two-dimensional medical image.

[0139] The reconstruction module 1703 is used to obtain a first transformation matrix between the coordinate system of the digitally reconstructed two-dimensional image and the coordinate system of the two-dimensional imaging device based on the imaging parameters of the digitally reconstructed two-dimensional image obtained by digital image reconstruction of the three-dimensional model, as well as the imaging parameters of the first two-dimensional medical imaging and the second two-dimensional medical imaging respectively.

[0140] The third acquisition module 1704 is used to acquire the planned pose of the target object in the coordinate system of the two-dimensional imaging device based on the initial pose of the target object in the three-dimensional model and the first transformation matrix.

[0141] The configuration module 1705 is used to convert the planned pose to the robot arm coordinate system to obtain the desired pose of the target object in the robot arm coordinate system.

[0142] In one embodiment, the first acquisition module 1701 includes:

[0143] The first acquisition submodule is used to acquire CT images containing the target object acquired before surgery;

[0144] The processing submodule is used to perform interpolation, noise reduction and contrast enhancement processing on the 3D medical images in sequence, and to perform 3D reconstruction based on the processed 3D medical images to obtain a 3D model constructed for the target object.

[0145] In one embodiment, the reconstruction module 1703 includes:

[0146] The registration submodule is used to register the digitally reconstructed two-dimensional image with the first two-dimensional medical image and the second two-dimensional medical image respectively, so as to obtain the digitally reconstructed two-dimensional image correspondingly registered with the first two-dimensional medical image and the second two-dimensional medical image respectively.

[0147] The determination submodule is used to determine the first transformation matrix based on the imaging parameters of the first two-dimensional medical imaging and the second two-dimensional medical imaging, as well as the digital reconstruction imaging parameters of the digitally reconstructed two-dimensional image corresponding to the first two-dimensional medical imaging and the digitally reconstructed two-dimensional image corresponding to the second two-dimensional medical imaging.

[0148] In one embodiment, the two-dimensional imaging device includes a C-arm and an X-ray tube; when a first two-dimensional medical image is obtained through imaging via the X-ray tube, the C-arm is in a first posture; when a second two-dimensional medical image is obtained through imaging via the X-ray tube, the C-arm is in a second posture.

[0149] In one embodiment, the registration submodule includes:

[0150] A reconstruction unit is used to perform digital image reconstruction of a three-dimensional model to obtain at least one digitally reconstructed two-dimensional image.

[0151] The registration unit is used to register the first two-dimensional medical image and the second two-dimensional medical image with the digitally reconstructed two-dimensional image, respectively. If the first two-dimensional medical image or the second two-dimensional medical image has not determined the corresponding registered digitally reconstructed two-dimensional image, the digital image reconstruction parameters are updated and the three-dimensional model is digitally reconstructed. Multiple digitally reconstructed two-dimensional images are obtained again and re-registered. The above update, reconstruction and registration process is repeated until the first two-dimensional medical image and the second two-dimensional medical image have both determined the corresponding registered digitally reconstructed two-dimensional images, and the corresponding registered digitally reconstructed two-dimensional images of the first two-dimensional medical image and the second two-dimensional medical image are obtained.

[0152] In one embodiment, the reconstruction unit includes:

[0153] The first acquisition subunit is used to acquire digital reconstruction imaging parameters, which include the location of the simulated light source and the projection angle of the simulated light source onto the three-dimensional model.

[0154] The computational subunit is used to emit light rays from a simulated light source into the 3D model according to the projection angle and its position, and to calculate the coordinates of the intersection point between the light rays emitted by the simulated light source and the 3D model when they pass through the 3D model.

[0155] The second acquisition subunit is used to acquire the gray values ​​of the intersection coordinates of each ray after trilinear interpolation, and to accumulate the gray values ​​of the intersection coordinates of each ray.

[0156] The third acquisition subunit is used to assign the accumulated gray values ​​of each ray to the intersection point between each ray and the digital imaging plane, thereby obtaining the digital reconstructed two-dimensional image corresponding to the digital reconstruction imaging parameters.

[0157] Each module in the aforementioned image-based registration device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0158] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 18 As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements an image-based registration method. The display screen can be an LCD screen or an e-ink display. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0159] Those skilled in the art will understand that Figure 18The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0160] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0161] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0162] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0163] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0164] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0165] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0166] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An image-based registration method, characterized in that, A registration system comprising a two-dimensional imaging device and a robotic arm, wherein the two-dimensional imaging device and the robotic arm are fixedly connected; the method includes: Based on preoperative 3D medical images, a 3D model constructed for the target object is obtained; Acquire a first two-dimensional medical image and a second two-dimensional medical image of the target object, wherein the posture of the two-dimensional imaging device when acquiring the first two-dimensional medical image is orthogonal to the posture when acquiring the second two-dimensional medical image. Based on the imaging parameters of the digitally reconstructed two-dimensional image obtained by digital image reconstruction of the three-dimensional model, and the imaging parameters of the first two-dimensional medical imaging and the second two-dimensional medical imaging respectively, a first transformation matrix between the coordinate system of the digitally reconstructed two-dimensional image and the coordinate system of the two-dimensional imaging device is obtained. Based on the initial pose of the target object in the three-dimensional model and the first transformation matrix, the planned pose of the target object in the coordinate system of the two-dimensional imaging device is obtained. The planned pose is transformed into the robot arm coordinate system to obtain the desired pose of the target object in the robot arm coordinate system; The step of obtaining a first transformation matrix between the coordinate system of the digitally reconstructed two-dimensional image and the coordinate system of the two-dimensional imaging device based on the imaging parameters of the digitally reconstructed two-dimensional image obtained by digital image reconstruction of the three-dimensional model, and the imaging parameters of the first two-dimensional medical imaging and the second two-dimensional medical imaging respectively, includes: The digitally reconstructed 2D images are registered with the first 2D medical image and the second 2D medical image, respectively, to obtain digitally reconstructed 2D images corresponding to the first 2D medical image and the second 2D medical image, respectively; there are multiple digitally reconstructed 2D images; the digitally reconstructed 2D image corresponding to the first 2D medical image refers to the digitally reconstructed 2D image corresponding to the first 2D medical image and each digitally reconstructed 2D image with a similarity greater than a preset similarity and the largest similarity; the digitally reconstructed 2D image corresponding to the second 2D medical image refers to the digitally reconstructed 2D image corresponding to the second 2D medical image and each digitally reconstructed 2D image with a similarity greater than a preset similarity and the largest similarity. The first transformation matrix is ​​determined based on the imaging parameters of the first two-dimensional medical imaging and the second two-dimensional medical imaging, as well as the digital reconstruction imaging parameters of the digitally reconstructed two-dimensional images registered accordingly to the first two-dimensional medical imaging and the digitally reconstructed two-dimensional images registered accordingly to the second two-dimensional medical imaging.

2. The method according to claim 1, characterized in that, The acquisition of a 3D model of the target object based on preoperative 3D medical images includes: The three-dimensional medical image is sequentially subjected to interpolation, noise reduction, and contrast enhancement. Based on the processed three-dimensional medical image, a three-dimensional reconstruction is performed to obtain a three-dimensional model constructed for the target object.

3. The method according to claim 1, characterized in that, The two-dimensional imaging device includes a C-arm and an X-ray tube; when the first two-dimensional medical image is obtained through imaging via the X-ray tube, the C-arm is in a first posture; when the second two-dimensional medical image is obtained through imaging via the X-ray tube, the C-arm is in a second posture.

4. The method according to claim 1, characterized in that, The step of registering the digitally reconstructed two-dimensional image with the first two-dimensional medical image and the second two-dimensional medical image respectively to obtain digitally reconstructed two-dimensional images correspondingly registered with the first two-dimensional medical image and the second two-dimensional medical image includes: The three-dimensional model is digitally reconstructed to obtain at least one digitally reconstructed two-dimensional image; The first two-dimensional medical image and the second two-dimensional medical image are respectively registered with the digitally reconstructed two-dimensional image. If the first two-dimensional medical image or the second two-dimensional medical image does not determine the corresponding registered digitally reconstructed two-dimensional image, the digital image reconstruction parameters are updated and the three-dimensional model is digitally reconstructed. Multiple digitally reconstructed two-dimensional images are obtained again and re-registered. The above update, reconstruction and registration process is repeated until the first two-dimensional medical image and the second two-dimensional medical image both determine the corresponding registered digitally reconstructed two-dimensional images, and the corresponding registered digitally reconstructed two-dimensional images of the first two-dimensional medical image and the second two-dimensional medical image are obtained.

5. The method according to claim 4, characterized in that, The step of performing digital image reconstruction on the three-dimensional model to obtain at least one digitally reconstructed two-dimensional image includes: Obtain digital reconstruction imaging parameters, including the location of the simulated light source and the projection angle of the simulated light source onto the three-dimensional model; According to the projection angle and the location, light rays are emitted from the simulated light source towards the three-dimensional model, and the coordinates of the intersection point between the light rays emitted from the simulated light source and the three-dimensional model when the light rays pass through the three-dimensional model are calculated; The grayscale values ​​of the intersection points of each ray are obtained by trilinear interpolation, and the grayscale values ​​of the intersection points of each ray are accumulated. The accumulated gray values ​​of each ray are assigned to the intersection point between each ray and the digital imaging plane to obtain the digital reconstructed two-dimensional image corresponding to the digital reconstruction imaging parameters.

6. The method according to claim 3, characterized in that, The first posture is orthogonal to the second posture.

7. An image-based registration device, applied in a registration system comprising a two-dimensional imaging device and a robotic arm, wherein the two-dimensional imaging device and the robotic arm are fixedly connected; characterized in that, The device includes: The first acquisition module is used to acquire a three-dimensional model of the target object based on the three-dimensional medical images taken before surgery. The second acquisition module is used to acquire the first two-dimensional medical image and the second two-dimensional medical image of the target object. The posture of the two-dimensional imaging device when acquiring the first two-dimensional medical image is orthogonal to the posture when acquiring the second two-dimensional medical image. The reconstruction module is used to obtain a first transformation matrix between the coordinate system of the digitally reconstructed two-dimensional image and the coordinate system of the two-dimensional imaging device based on the imaging parameters of the digitally reconstructed two-dimensional image obtained by digital image reconstruction of the three-dimensional model, and the imaging parameters of the first two-dimensional medical imaging and the second two-dimensional medical imaging respectively. The third acquisition module is used to acquire the planned pose of the target object in the coordinate system of the two-dimensional imaging device based on the initial pose of the target object in the three-dimensional model and the first transformation matrix. A configuration module is used to convert the planned pose to the robot arm coordinate system to obtain the desired pose of the target object in the robot arm coordinate system; The reconstruction module is further configured to register the digitally reconstructed two-dimensional image with the first two-dimensional medical image and the second two-dimensional medical image respectively, to obtain digitally reconstructed two-dimensional images correspondingly registered with the first two-dimensional medical image and the second two-dimensional medical image respectively; there are multiple digitally reconstructed two-dimensional images; the digitally reconstructed two-dimensional image correspondingly registered with the first two-dimensional medical image refers to the digitally reconstructed two-dimensional image corresponding to the first two-dimensional medical image and each digitally reconstructed two-dimensional image with a similarity greater than a preset similarity and the largest similarity; the digitally reconstructed two-dimensional image correspondingly registered with the second two-dimensional medical image refers to the digitally reconstructed two-dimensional image corresponding to the second two-dimensional medical image and each digitally reconstructed two-dimensional image with a similarity greater than a preset similarity and the largest similarity; a first transformation matrix is ​​determined based on the imaging parameters of the first two-dimensional medical image and the second two-dimensional medical image respectively, and the digitally reconstructed imaging parameters of the digitally reconstructed two-dimensional images correspondingly registered with the first two-dimensional medical image and the digitally reconstructed two-dimensional images correspondingly registered with the second two-dimensional medical image respectively.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is run on a computer, it causes the computer to perform the method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is run on a computer, it causes the computer to perform the method of any one of claims 1 to 6.

Images