Calibration method, surgical navigation method, medical device and computer program product

Abstract

The application is suitable for the medical technical field, and provides a calibration method, a surgical navigation method, medical equipment and a computer program product. The calibration method comprises: acquiring first medical images and second medical images obtained by a C-arm imaging system in two shooting poses; acquiring third medical images and fourth medical images obtained by the C-arm imaging system in the two shooting poses when a surgical instrument is in an imaging field of view; taking a C-arm imaging system coordinate system corresponding to the first medical images as a world coordinate system, determining first coordinates of instrument feature points of the surgical instrument in the world coordinate system in the third medical images and the fourth medical images; and determining first conversion parameters between a mechanical arm coordinate system of a mechanical arm and the C-arm imaging system coordinate system according to the first coordinates and second coordinates of the instrument feature points in the mechanical arm coordinate system. Embodiments of the application can complete calibration in real time without a calibration target, so that the conversion parameters are more accurate.

Description

Technical Field

[0001] This application belongs to the field of medical technology, and in particular relates to a calibration method, a surgical navigation method, a medical device, and a computer program product. Background Technology

[0002] When using a C-arm for surgical navigation, calibration of the C-arm imaging system is usually required. The calibration of the C-arm imaging system can be approximated as camera calibration, typically requiring the use of a calibration target. Coordinate transformation parameters are obtained through online calibration (calibrating the target on-site by photographing it) or offline calibration (extensive calibration with the target before use, followed by interpolation using a lookup table during use). On the one hand, calibration targets require specialized design, fabrication, and measurement, and their use is subject to certain constraints, increasing both the time and difficulty of use. On the other hand, the transformation parameters obtained through interpolation during offline calibration often contain errors. Summary of the Invention

[0003] This application provides a calibration method, a surgical navigation method, a medical device, and a computer program product that can complete calibration in real time without a calibration target, making the conversion parameters more accurate.

[0004] A first aspect of this application provides a calibration method, comprising: acquiring a first medical image and a second medical image obtained by a C-arm imaging system capturing images of a target object in a first shooting pose and a second shooting pose, respectively; determining a first coordinate transformation matrix between the first shooting pose and the second shooting pose of the C-arm imaging system based on the first medical image and the second medical image; using the C-arm imaging system coordinate system corresponding to the first medical image as the world coordinate system, acquiring a first projection matrix of the first medical image; obtaining a second projection matrix of the second medical image based on the first projection matrix and the first coordinate transformation matrix; acquiring a third medical image captured by the C-arm imaging system in a third shooting pose and a fourth medical image captured in a fourth shooting pose when the surgical instrument is located in the imaging field of view of the C-arm imaging system; determining a first coordinate of the instrument feature point of the surgical instrument in the third medical image and the fourth medical image in the world coordinate system based on the first coordinate and the second projection matrix; and determining a first transformation parameter between the robotic arm coordinate system and the C-arm imaging system coordinate system based on the first coordinate and the second coordinate of the instrument feature point in the robotic arm coordinate system.

[0005] In some embodiments of the first aspect, determining the first coordinate transformation matrix between the first shooting pose and the second shooting pose of the C-arm imaging system based on the first medical image and the second medical image includes: determining matching feature point pairs in the first medical image and the second medical image; and determining the first coordinate transformation matrix of the C-arm imaging system coordinate system of the second medical image relative to the C-arm imaging system coordinate system of the first medical image based on the pixel positions of the feature point pairs.

[0006] In some embodiments of the first aspect, the third shooting pose is the same as the first shooting pose, and the fourth shooting pose is the same as the second shooting pose; determining the first coordinates of the instrument feature points of the surgical instrument in the third medical image and the fourth medical image in the world coordinate system according to the first projection matrix and the second projection matrix includes: obtaining at least three third feature points of the surgical instrument in the third medical image, obtaining at least three fourth feature points of the surgical instrument in the fourth medical image corresponding to the third feature points; and obtaining the first coordinates of the instrument feature points of the surgical instrument in the world coordinate system according to the first projection matrix, the second projection matrix, the third feature points, and the fourth feature points.

[0007] In some embodiments of the first aspect, determining the first coordinates of the instrument feature points of the surgical instrument in the third medical image and the fourth medical image in the world coordinate system based on the first projection matrix and the second projection matrix includes: acquiring at least three third feature points of the surgical instrument in the third medical image; acquiring at least three fourth feature points of the surgical instrument in the fourth medical image corresponding to the third feature points; registering the third medical image with the first medical image to obtain a first registration matrix; registering the fourth medical image with the second medical image to obtain a second registration matrix; and obtaining the first coordinates of the instrument feature points of the surgical instrument in the world coordinate system based on the first registration matrix, the second registration matrix, the first projection matrix, the second projection matrix, the third feature points, and the fourth feature points.

[0008] In some embodiments of the first aspect, the second shooting pose is obtained by controlling the C-arm imaging system to translate a preset distance from the first shooting pose along a preset direction.

[0009] In some embodiments of the first aspect, the preset direction is the direction perpendicular to the sagittal plane of the target object.

[0010] A second aspect of this application provides a calibration method, comprising: when a surgical instrument is located in the imaging field of view of a C-arm imaging system, acquiring a fifth medical image and a sixth medical image obtained by the C-arm imaging system capturing images of a target object in a first shooting pose and a second shooting pose, respectively; determining a first coordinate transformation matrix between the first shooting pose and the second shooting pose of the C-arm imaging system based on the fifth medical image and the sixth medical image; using the C-arm imaging system coordinate system corresponding to the fifth medical image as the world coordinate system, acquiring a first projection matrix of the fifth medical image; obtaining a second projection matrix of the sixth medical image based on the first projection matrix and the first coordinate transformation matrix; determining a first coordinate of the instrument feature point of the surgical instrument in the fifth medical image and the sixth medical image in the world coordinate system based on the first projection matrix and the second projection matrix; and determining a first transformation parameter between the robotic arm coordinate system and the C-arm imaging system coordinate system based on the first coordinate and the second coordinate of the instrument feature point in the robotic arm coordinate system.

[0011] In some embodiments of the second aspect, determining the first coordinate transformation matrix between the first shooting pose and the second shooting pose of the C-arm imaging system based on the fifth medical image and the sixth medical image includes: determining matching feature point pairs in the fifth medical image and the sixth medical image; and determining the first coordinate transformation matrix of the C-arm imaging system coordinate system of the sixth medical image relative to the C-arm imaging system coordinate system of the fifth medical image based on the pixel positions of the feature point pairs.

[0012] In some embodiments of the second aspect, determining the first coordinates of the instrument feature points of the surgical instrument in the fifth medical image and the sixth medical image in the world coordinate system based on the first projection matrix and the second projection matrix includes: obtaining at least three third feature points of the surgical instrument in the fifth medical image, obtaining at least three fourth feature points of the surgical instrument in the sixth medical image corresponding to the third feature points; and obtaining the first coordinates of the instrument feature points of the surgical instrument in the world coordinate system based on the first projection matrix, the second projection matrix, the third feature points, and the fourth feature points.

[0013] A third aspect of this application provides a surgical navigation method, comprising: acquiring a first conversion parameter, wherein the first conversion parameter is obtained according to any one of the calibration methods of the first aspect or any one of the calibration methods of the second aspect; displaying a target medical image captured by a C-arm imaging system; and displaying the surgical instrument in the target medical image according to the real-time coordinates of the surgical instrument in the robotic arm coordinate system and the first conversion parameter.

[0014] In some embodiments of the third aspect, the target medical image is the first medical image and / or the second medical image.

[0015] In some embodiments of the third aspect, before displaying the target medical image captured by the C-arm imaging system, the method further includes: controlling the C-arm imaging system to re-capture a medical image as the target medical image; the step of displaying the surgical instrument in the target medical image based on the real-time coordinates of the surgical instrument in the robotic arm coordinate system and the first transformation parameter includes: determining a second coordinate transformation matrix between the target medical image and the first medical image, and displaying the surgical instrument in the target medical image based on the real-time coordinates, the first transformation parameter, and the second coordinate transformation matrix.

[0016] A calibration apparatus provided in the fourth aspect of this application includes: a first image acquisition unit, configured to acquire a first medical image and a second medical image obtained by a C-arm imaging system capturing images of a target object in a first shooting pose and a second shooting pose, respectively; a coordinate transformation matrix determination unit, configured to determine a first coordinate transformation matrix between the first shooting pose and the second shooting pose of the C-arm imaging system based on the first medical image and the second medical image; a first projection matrix acquisition unit, configured to acquire a first projection matrix of the first medical image by using the coordinate system of the C-arm imaging system corresponding to the first medical image as the world coordinate system; and a second projection matrix acquisition unit, configured to acquire a first projection matrix of the first medical image based on the first projection matrix and the first coordinate transformation matrix. The system obtains a second projection matrix for the second medical image; a second image acquisition unit is used to acquire a third medical image captured by the C-arm imaging system in a third shooting pose and a fourth medical image captured in a fourth shooting pose when the surgical instrument is located in the imaging field of view of the C-arm imaging system; a world coordinate acquisition unit is used to determine the first coordinates of the instrument feature points of the surgical instrument in the third medical image and the fourth medical image in the world coordinate system based on the first projection matrix and the second projection matrix; a calibration unit is used to determine the first transformation parameter between the robotic arm coordinate system and the C-arm imaging system coordinate system based on the first coordinates and the second coordinates of the instrument feature points in the robotic arm's robotic arm coordinate system.

[0017] A calibration device provided in the fifth aspect of this application includes: a third image acquisition unit, configured to acquire a fifth medical image and a sixth medical image obtained by the C-arm imaging system capturing images of a target object in a first shooting pose and a second shooting pose, respectively, when the surgical instrument is located in the imaging field of view of the C-arm imaging system; a coordinate transformation matrix determination unit, configured to determine a first coordinate transformation matrix between the first shooting pose and the second shooting pose of the C-arm imaging system based on the fifth medical image and the sixth medical image; a first projection matrix acquisition unit, configured to acquire a first projection matrix of the fifth medical image by using the C-arm imaging system coordinate system corresponding to the fifth medical image as the world coordinate system; a second projection matrix acquisition unit, configured to obtain a second projection matrix of the sixth medical image based on the first projection matrix and the first coordinate transformation matrix; a world coordinate acquisition unit, configured to determine the first coordinates of the instrument feature points of the surgical instrument in the fifth medical image and the sixth medical image in the world coordinate system based on the first projection matrix and the second projection matrix; and a calibration unit, configured to determine a first transformation parameter between the robotic arm coordinate system and the C-arm imaging system coordinate system based on the first coordinates and the second coordinates of the instrument feature points in the robotic arm coordinate system.

[0018] A surgical navigation device provided in the sixth aspect of this application includes: a calibration unit for acquiring a first conversion parameter, wherein the first conversion parameter is obtained according to the calibration method described in any one of the first aspects; an image display unit for displaying a target medical image captured by a C-arm imaging system; and an instrument display unit for displaying the surgical instrument in the target medical image based on the real-time coordinates of the surgical instrument in the robotic arm coordinate system and the first conversion parameter.

[0019] A seventh aspect of this application provides a medical device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the calibration method as described in any of the first aspects; or, when the processor executes the computer program, it implements the steps of the calibration method as described in any of the second aspects; or, when the processor executes the computer program, it implements the steps of the surgical navigation method as described in any of the third aspects.

[0020] An eighth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the calibration method described in the first aspect; or, the computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the calibration method described in the second aspect; or, the computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the surgical navigation method described in the third aspect.

[0021] A ninth aspect of this application provides a computer program product, including a computer program that, when run, causes the calibration method of the first aspect to be executed, or causes the calibration method of the second aspect to be executed, or causes the surgical navigation method of the third aspect to be executed.

[0022] In the embodiments of this application, a first medical image and a second medical image are obtained by the C-arm imaging system capturing the target object in a first shooting pose and a second shooting pose, respectively. A first coordinate transformation matrix between the first shooting pose and the second shooting pose is determined. The coordinate system of the C-arm imaging system corresponding to the first medical image is used as the world coordinate system. A first projection matrix and a second projection matrix of the first medical image are obtained. When the surgical instrument is located in the imaging field of view of the C-arm imaging system, a third medical image captured in a third shooting pose and a fourth medical image captured in a fourth shooting pose are obtained. Subsequently, based on the first projection matrix and the second projection matrix, the first coordinates of the instrument feature points of the surgical instrument in the third and fourth medical images in the world coordinate system are determined. Based on the first coordinates and the second coordinates of the instrument feature points in the robotic arm's robotic arm coordinate system, a first transformation parameter between the robotic arm coordinate system and the C-arm imaging system coordinate system is determined. This method allows for calibration to be completed in real time during the medical image capture process, without the need for a calibration target. Compared with the transformation parameters obtained by interpolation, this method can obtain more accurate transformation parameters based on the on-site installation conditions of the C-arm imaging system. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a schematic diagram illustrating the implementation process of a calibration method provided in an embodiment of this application;

[0025] Figure 2 This is a schematic diagram of capturing a first medical image provided in an embodiment of this application;

[0026] Figure 3 This is a schematic diagram of capturing a second medical image provided in an embodiment of this application;

[0027] Figure 4 This is a schematic diagram illustrating the specific implementation process of determining the first coordinate transformation matrix provided in an embodiment of this application;

[0028] Figure 5 This application provides an embodiment of a schematic diagram illustrating the capture of a third medical image;

[0029] Figure 6 This application provides a schematic diagram of the fourth medical image capture method in its embodiments;

[0030] Figure 7 This is a schematic diagram illustrating the implementation process of another calibration method provided in this application embodiment;

[0031] Figure 8 This is a schematic diagram illustrating the implementation process of the surgical navigation method provided in the embodiments of this application;

[0032] Figure 9 This is a schematic diagram of the structure of a calibration device provided in an embodiment of this application;

[0033] Figure 10 This is a schematic diagram of another calibration device provided in an embodiment of this application;

[0034] Figure 11 This is a schematic diagram of the structure of a surgical navigation device provided in an embodiment of this application;

[0035] Figure 12 This is a schematic diagram of the structure of the medical device provided in the embodiments of this application. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are protected by this application.

[0037] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0038] In the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0039] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0040] When using a C-arm for surgical navigation, calibration of the C-arm imaging system is usually required. The calibration of the C-arm imaging system can be approximated as camera calibration, typically requiring the use of a calibration target. Coordinate transformation parameters are obtained through online calibration (calibrating the target on-site by photographing it) or offline calibration (extensive calibration with the target before use, followed by interpolation using a lookup table during use). On the one hand, calibration targets require specialized design, fabrication, and measurement, and their use is subject to certain constraints, increasing both the time and difficulty of use. On the other hand, the transformation parameters obtained through interpolation during offline calibration often contain errors.

[0041] In view of this, this application proposes a calibration method that can complete calibration in real time without a calibration target, making the conversion parameters more accurate.

[0042] To illustrate the technical solution of this application, specific embodiments are described below.

[0043] Please refer to Figure 1 , Figure 1 The illustration shows a schematic diagram of the implementation process of a calibration method provided in an embodiment of this application. This method can be applied to medical devices, which can be computers, servers or other intelligent devices, and this application does not limit the scope of application.

[0044] In some embodiments of this application, the aforementioned medical device can be connected to a C-arm imaging system and a robotic arm. The C-arm imaging system generates X-rays through an X-ray tube on a C-frame, and the receiving end receives the X-rays to obtain medical images. Medical images can refer to X-ray images. The end effector of the robotic arm can be equipped with surgical instruments for performing surgical procedures.

[0045] Specifically, the above calibration method may include the following steps S101 to S107.

[0046] Step S101: Obtain the first medical image and the second medical image obtained by the C-arm imaging system capturing the target object in the first shooting pose and the second shooting pose, respectively.

[0047] The first and second imaging poses are different. By controlling the C-arm imaging system to capture images of the target object in both the first and second imaging poses, two medical images can be obtained, which are the first medical image and the second medical image. The first and second medical images can each correspond to one imaging pose.

[0048] In embodiments of this application, the target object can be a person, animal, or other subject to be photographed, and this application does not impose any limitations on this. In some embodiments, the target object can be a patient. During surgical navigation of a patient, a first medical image and a second medical image can be obtained by taking pictures of the patient in a first shooting pose and a second shooting pose, respectively, thereby integrating the calibration process into the surgical navigation process.

[0049] In some embodiments of this application, there are matching object feature points between the first medical image and the second medical image. Object feature points are feature points on the target object. These object feature points can be feature points located at the target location (e.g., on blocks, edges, corners), and their number is less than a quantity threshold (i.e., relatively rare, much less than the number of pixels) to facilitate subsequent identification. For example, object feature points may include, but are not limited to: the superior and inferior endplates and spinous and transverse processes of the vertebral bodies of the spine, the long and short axes of the obturator foramen of the pelvis, and the lesser trochanter of the femur.

[0050] It should be noted that this application does not restrict the order in which the first medical image and the second medical image are captured. That is to say, when the C-arm imaging system captures images in the first shooting pose, the resulting medical image can be either the first medical image or the second medical image.

[0051] Step S102: Determine the first coordinate transformation matrix between the first and second shooting poses of the C-arm imaging system based on the first medical image and the second medical image.

[0052] Specifically, since the first and second imaging poses are known, the pose change between them can be calculated. Based on the matching point pairs in the first and second medical images, a first coordinate transformation matrix between the first and second imaging poses can be calculated. This first coordinate transformation matrix, used for coordinate transformation between the C-arm imaging system coordinate system corresponding to the first and second medical images, can be composed of a rotation matrix R and a translation matrix t.

[0053] Step S103: Using the coordinate system of the C-arm imaging system corresponding to the first medical image as the world coordinate system, obtain the first projection matrix of the first medical image.

[0054] In the embodiments of this application, the projection matrix is ​​the product of the intrinsic parameter matrix and the extrinsic parameter matrix. The intrinsic parameter matrix of the C-arm imaging system is related to the structure of the C-arm itself and represents prior information. Since the coordinate system of the C-arm imaging system corresponding to the first medical image is used as the world coordinate system, the rotation matrix R in the extrinsic parameter matrix corresponding to the first medical image is a 3×3 identity matrix, and the translation matrix t is a 3×1 zero matrix. Multiplying the intrinsic parameter matrix and the extrinsic parameter matrix corresponding to the first medical image yields the first projection matrix of the first medical image.

[0055] Step S104: Obtain the second projection matrix of the second medical image based on the first projection matrix and the first coordinate transformation matrix.

[0056] In the embodiments of this application, the first projection matrix can be transformed based on the first coordinate transformation matrix between the first and second shooting poses to obtain the second projection matrix of the second medical image. Specifically, when the coordinate system of the C-arm imaging system corresponding to the first medical image is taken as the world coordinate system, the extrinsic parameter matrix corresponding to the second medical image is the first coordinate transformation matrix. Multiplying the intrinsic parameter matrix and the extrinsic parameter matrix corresponding to the second medical image yields the second projection matrix of the second medical image.

[0057] Step S105: When the surgical instruments are in the imaging field of the C-arm imaging system, acquire the third medical image captured by the C-arm imaging system in the third shooting pose and the fourth medical image captured in the fourth shooting pose.

[0058] In embodiments of this application, by controlling the robotic arm, surgical instruments at the end of the robotic arm can be moved into the imaging field of view of the C-arm imaging system. Preferably, the surgical instruments can be moved near the patient area so that the aforementioned object feature points are not obstructed in the imaging field of view. Subsequently, by taking pictures in the third and fourth shooting poses respectively using the C-arm imaging system, third and fourth medical images can be obtained.

[0059] The third and fourth medical images correspond to the first and second medical images, respectively. Specifically, the third medical image can be taken in a pose that is the same as or similar to the first shooting pose, and the fourth medical image can be taken in a pose that is the same as or similar to the second shooting pose.

[0060] Step S106: Based on the first projection matrix and the second projection matrix, determine the first coordinates of the instrument feature points of the surgical instruments in the third and fourth medical images in the world coordinate system.

[0061] Here, instrument feature points refer to feature points on surgical instruments. Taking the coordinate system of the C-arm imaging system corresponding to the first medical image as the world coordinate system, the first coordinates of the instrument feature points on the surgical instruments in the world coordinate system can be obtained according to the first coordinate transformation matrix, the first projection matrix, and the second projection matrix.

[0062] Step S107: Based on the first coordinates and the second coordinates of the instrument feature points in the robotic arm coordinate system, determine the first transformation parameter between the robotic arm coordinate system and the C-arm imaging system coordinate system.

[0063] In some embodiments of this application, since the surgical instrument is moved by the movement of a robotic arm, the position of the surgical instrument in the robotic arm's coordinate system is known. Since the instrument feature point is located on the surgical instrument, the second coordinate of the instrument feature point in the robotic arm's coordinate system is prior information. Based on the first coordinate of the instrument feature point in the world coordinate system and the second coordinate of the instrument feature point in the robotic arm's coordinate system, the transformation relationship between the world coordinate system and the robotic arm coordinate system can be determined. Since the world coordinate system is the coordinate system of the C-arm imaging system corresponding to the first medical image, a first transformation parameter between the robotic arm coordinate system and the C-arm imaging system coordinate system can be obtained. This first transformation parameter can be used to convert the coordinates of the robotic arm coordinate system to the coordinates of the C-arm imaging system coordinate system, or vice versa.

[0064] It should be noted that the above-mentioned robotic arm coordinate system can be the base coordinate system of the robotic arm (i.e., a coordinate system established with the mounting base of the robotic arm as the reference), or a coordinate system established with other parts of the robotic arm as the reference. This application does not impose any restrictions on this.

[0065] In the embodiments of this application, a first medical image and a second medical image are obtained by the C-arm imaging system capturing the target object in a first shooting pose and a second shooting pose, respectively. A first coordinate transformation matrix between the first shooting pose and the second shooting pose is determined. The coordinate system of the C-arm imaging system corresponding to the first medical image is used as the world coordinate system. A first projection matrix and a second projection matrix of the first medical image are obtained. When the surgical instrument is located in the imaging field of view of the C-arm imaging system, a third medical image captured in a third shooting pose and a fourth medical image captured in a fourth shooting pose are obtained. Subsequently, based on the first projection matrix and the second projection matrix, the first coordinates of the instrument feature points of the surgical instrument in the third and fourth medical images in the world coordinate system are determined. Based on the first coordinates and the second coordinates of the instrument feature points in the robotic arm's robotic arm coordinate system, a first transformation parameter between the robotic arm coordinate system and the C-arm imaging system coordinate system is determined. This method allows for calibration to be completed in real time during the medical image capture process, without the need for a calibration target. Compared with the transformation parameters obtained by interpolation, this method can obtain more accurate transformation parameters based on the on-site installation conditions of the C-arm imaging system.

[0066] In some embodiments of this application, the second shooting pose is obtained by controlling the C-arm imaging system to translate the first shooting pose along a preset direction by a preset distance.

[0067] For details, please refer to Figure 2 and Figure 3 The system can control the C-arm imaging system to capture a first medical image in the area with obvious object feature points in the first shooting pose. With the target object stationary, the C-arm imaging system can be translated a preset distance along a preset direction to ensure that the object feature points in the first shooting pose also appear in the imaging field of view, and a second medical image can be obtained by capturing the area with obvious object feature points.

[0068] In some embodiments of this application, the aforementioned preset direction can be the horizontal direction of the C-arm imaging system. Specifically, when the C-arm imaging system is used to scan the coronal plane of the target object, the preset direction is the vertical direction of the sagittal plane of the target object. In this way, by moving a small preset distance, the requirement for the object's feature points to appear in the imaging field of view can be met, and differences can be created between the two medical images, that is, the pixel positions of the same object feature point differ between the two medical images.

[0069] In some embodiments of this application, such as Figure 4 As shown, determining the first coordinate transformation matrix between the first and second shooting poses of the C-arm imaging system based on the first and second medical images may include steps S401 to S402.

[0070] Step S401: Identify matching feature point pairs in the first medical image and the second medical image.

[0071] Specifically, feature points can be extracted from the first and second medical images respectively, and the feature points on the two medical images can be matched to obtain matching feature point pairs. Each feature point pair includes a first feature point on the first medical image and a second feature point on the second medical image corresponding to the first feature point. The first and second feature points can be object feature points on the target object. The number of feature point pairs can be at least eight pairs.

[0072] Step S402: Based on the pixel positions of the feature point pairs, determine the first coordinate transformation matrix of the C-arm imaging system coordinate system of the second medical image relative to the C-arm imaging system coordinate system of the first medical image.

[0073] Specifically, based on the pixel positions of the first feature point in the first medical image and the second feature point in the second medical image, the essential matrix E can be obtained using the eight-point method. The essential matrix E describes the relationship between the coordinates of a point in space within the camera coordinate system under different shooting poses. Based on the essential matrix E, the first coordinate transformation matrix of the C-arm imaging system coordinate system of the second medical image relative to the C-arm imaging system coordinate system of the first medical image can be obtained through singular value decomposition. This first coordinate transformation matrix can be represented as a rotation matrix R and a translation matrix t.

[0074] In some embodiments of this application, during the process of acquiring the third and fourth medical images captured by the C-arm imaging system, the third shooting pose may be the same as the first shooting pose, and the fourth shooting pose may be the same as the second shooting pose.

[0075] Specifically, such as Figure 5 and Figure 6 As shown, when the C-arm imaging system can record the first and second shooting poses, or can be precisely moved to the first and second shooting poses, the C-arm imaging system can be controlled to take pictures in the first and second shooting poses to obtain the third and fourth medical images, respectively.

[0076] At this point, the third medical image and the first medical image were captured from the same shooting pose, and the fourth medical image and the second medical image were captured from the same shooting pose. Based on the first projection matrix and the second projection matrix, determining the first coordinates of the surgical instrument feature points in the third and fourth medical images in the world coordinate system can include: obtaining at least three third feature points of the surgical instrument in the third medical image, and obtaining at least three fourth feature points of the surgical instrument in the fourth medical image corresponding to the third feature points; and obtaining the first coordinates of the surgical instrument feature points in the world coordinate system based on the first projection matrix, the second projection matrix, the third feature points, and the fourth feature points.

[0077] Among them, the third and fourth feature points are both instrument feature points of the surgical instrument. At least three of the at least three third feature points are non-collinear. At least three of the at least three fourth feature points are non-collinear. At least three pairs of matching feature points can be formed between the at least three third feature points and the at least three fourth feature points.

[0078] Since the third shooting pose is the same as the first shooting pose, and the fourth shooting pose is the same as the second shooting pose, the pixel position of the third feature point in the third medical image can be directly used as the pixel position of the third feature point in the first medical image; similarly, the pixel position of the fourth feature point in the fourth medical image can be directly used as the pixel position of the fourth feature point in the second medical image. In other words, the first projection matrix of the first medical image is the projection matrix of the third medical image; and the first projection matrix of the second medical image is the projection matrix of the fourth medical image. Therefore, given that the first projection matrix of the first medical image and the second projection matrix of the second medical image are both known, the pixel positions of the third and fourth feature points in the third and fourth medical images can be used to simultaneously solve for the first coordinates of the surgical instrument feature points in the world coordinate system.

[0079] This method eliminates the need to register the third medical image with the first medical image and the fourth medical image with the second medical image, resulting in higher computational efficiency, but it requires higher precision in the shooting pose.

[0080] In some other embodiments of this application, after obtaining the third medical image and the fourth medical image, the method may further include: registering the third medical image with the first medical image; and registering the fourth medical image with the second medical image.

[0081] Specifically, based on the first and second projection matrices, determining the first coordinates of the surgical instrument feature points in the third and fourth medical images in the world coordinate system may include: obtaining at least three third feature points of the surgical instrument in the third medical image, and obtaining at least three fourth feature points of the surgical instrument in the fourth medical image corresponding to the third feature points. Registering the third and first medical images yields a first registration matrix. Registering the fourth and second medical images yields a second registration matrix. Based on the first and second registration matrices, the first and second projection matrices, the third and fourth feature points, the first coordinates of the surgical instrument feature points in the world coordinate system are obtained.

[0082] When the C-arm imaging system is manually controlled, it can be controlled to capture images in a third shooting pose similar to or the same as the first shooting pose, and in a fourth shooting pose similar to or the same as the second shooting pose. Image registration is then used to align the third and fourth medical images to the first and second shooting poses. For the third medical image corresponding to the first medical image, registration can be performed with the first medical image. For the fourth medical image corresponding to the second medical image, registration can be performed with the second medical image. Thus, after registration, the first registration matrix can be used to convert the pixel positions of the instrument feature points of the surgical instruments in the third medical image to the pixel positions in the first medical image; the second registration matrix can be used to convert the pixel positions of the instrument feature points of the surgical instruments in the fourth medical image to the pixel positions in the second medical image.

[0083] Using the first registration matrix, the pixel position of the third feature point in the third medical image can be converted to its pixel position in the first medical image. Using the second registration matrix, the pixel position of the fourth feature point in the fourth medical image can be converted to its pixel position in the second medical image. Now, with the first projection matrix of the first medical image and the second projection matrix of the second medical image both known, the first coordinates of the surgical instrument feature point in the world coordinate system can be obtained by simultaneously solving for the pixel positions of the third and fourth feature points in the first and second medical images.

[0084] After obtaining the first coordinates, the first transformation parameter between the robotic arm coordinate system and the C-arm imaging system coordinate system can be determined based on the first coordinates and the second coordinates of the instrument feature points in the robotic arm coordinate system.

[0085] In the embodiments of this application, calibration can be completed in real time without a calibration target by capturing four medical images. This method does not require the collection of a large amount of calibration target data under different shooting poses before use, nor does it require conversion parameters obtained through interpolation. More accurate conversion parameters can be obtained based on the on-site installation conditions and structural dimensions of the C-arm imaging system. Furthermore, by using the robotic arm coordinate system, the first conversion parameter can still be used when changing surgical instruments at the end of the robotic arm.

[0086] It should be noted that if the surgical instruments are within the imaging field of view of the C-arm imaging system when the first and second imaging positions are taken, calibration can be achieved directly using the two medical images.

[0087] For details, please refer to Figure 7 , Figure 7 The illustration shows a schematic diagram of the implementation process of another calibration method provided in this application embodiment, which can be applied to medical devices.

[0088] Specifically, the above calibration method may include the following steps S701 to S706.

[0089] Step S701: When the surgical instruments are in the imaging field of the C-arm imaging system, acquire the fifth and sixth medical images obtained by the C-arm imaging system taking pictures of the target object in the first and second shooting poses, respectively.

[0090] Step S702: Determine the first coordinate transformation matrix between the first shooting pose and the second shooting pose of the C-arm imaging system based on the fifth medical image and the sixth medical image.

[0091] Step S703: Using the coordinate system of the C-arm imaging system corresponding to the fifth medical image as the world coordinate system, obtain the first projection matrix of the fifth medical image.

[0092] Step S704: Obtain the second projection matrix of the sixth medical image based on the first projection matrix and the first coordinate transformation matrix.

[0093] Specifically, the fifth and sixth medical images are regarded as the first and second medical images, respectively. The implementation of steps S701 to S704 can be referred to the description of steps S101 to S104. The only difference is that the surgical instruments are located in the imaging field of the C-arm imaging system in the fifth and sixth medical images, which will not be elaborated in this application.

[0094] Step S705: Based on the first projection matrix and the second projection matrix, determine the first coordinates of the instrument feature points of the surgical instruments in the fifth and sixth medical images in the world coordinate system.

[0095] Step S706: Based on the first coordinates and the second coordinates of the instrument feature points in the robotic arm coordinate system, determine the first transformation parameter between the robotic arm coordinate system and the C-arm imaging system coordinate system.

[0096] Specifically, the fifth and sixth medical images are regarded as the third and fourth medical images, respectively. The implementation of steps S705 to S706 can be referred to the description of steps S106 to S107, which will not be repeated in this application.

[0097] Understandable, Figure 7 The calibration method shown is compared to Figure 1 The calibration method shown eliminates the need for two new medical images (the third and fourth) because the surgical instruments are within the imaging field of view of the C-arm imaging system when capturing the fifth and sixth medical images. Calibration can be completed in real-time, detached from the calibration target, by capturing these two images. This method avoids the need for extensive pre-collection of calibration target data under different shooting poses and the need for interpolated conversion parameters. More accurate conversion parameters can be obtained based on the on-site installation and structural dimensions of the C-arm imaging system. Furthermore, by using the robotic arm coordinate system, the first conversion parameter can still be used when changing the surgical instruments at the end of the robotic arm.

[0098] In some embodiments of this application, determining the first coordinate transformation matrix between the first and second shooting poses of the C-arm imaging system based on the fifth and sixth medical images may include: determining matching feature point pairs in the fifth and sixth medical images; and determining the first coordinate transformation matrix of the C-arm imaging system coordinate system of the sixth medical image relative to the C-arm imaging system coordinate system of the fifth medical image based on the pixel positions of the feature point pairs.

[0099] Specifically, based on the pixel positions of the first feature point in the fifth medical image and the second feature point in the sixth medical image, the essential matrix E can be obtained using the eight-point method. The essential matrix E describes the relationship between the coordinates of a point in space within the camera coordinate system under different shooting poses. Based on the essential matrix E, the first coordinate transformation matrix between the C-arm imaging system coordinate system of the sixth medical image and the C-arm imaging system coordinate system of the fifth medical image can be obtained through singular value decomposition. This first coordinate transformation matrix can be represented as a rotation matrix R and a translation matrix t.

[0100] In some embodiments of this application, determining the first coordinates of the instrument feature points of the surgical instruments in the fifth and sixth medical images in the world coordinate system based on the first projection matrix and the second projection matrix may include: obtaining at least three third feature points of the surgical instruments in the fifth medical image, obtaining at least three fourth feature points of the surgical instruments in the sixth medical image corresponding to the third feature points; and obtaining the first coordinates of the instrument feature points of the surgical instruments in the world coordinate system based on the first projection matrix, the second projection matrix, the third feature points, and the fourth feature points.

[0101] Specifically, given that the first projection matrix of the fifth medical image and the second projection matrix of the sixth medical image are both known, the first coordinates of the instrument feature points in the world coordinate system can be obtained by simultaneously solving the pixel positions of the third feature point in the fifth medical image and the fourth feature point in the sixth medical image.

[0102] Please refer to Figure 8 , Figure 8 The illustration shows a schematic flowchart of a surgical navigation method provided in an embodiment of this application. This method can be applied to medical devices, which can be computers, servers, or other intelligent devices, and this application does not limit the scope of application.

[0103] Specifically, the surgical navigation method described above may include the following steps S801 to S803.

[0104] Step S801: Obtain the first conversion parameter.

[0105] The method for obtaining the first conversion parameter can be found by referring to... Figures 1 to 6 The calibration method shown is explained in the instructions, or refer to the following: Figure 7 The calibration method shown is explained in detail in this application.

[0106] Step S802: Display the target medical image captured by the C-arm imaging system.

[0107] The target medical image can be any one or more medical images captured by a C-arm imaging system.

[0108] Step S803: Display the surgical instruments in the target medical image based on the real-time coordinates of the surgical instruments in the robotic arm coordinate system and the first transformation parameters.

[0109] In embodiments of this application, the surgical instrument can be adjusted to any position during surgery via a robotic arm, and the real-time coordinates of the surgical instrument in the robotic arm coordinate system can be obtained according to the control commands of the robotic arm. Based on the first transformation parameter, the real-time coordinates of the surgical instrument in the robotic arm coordinate system can be converted into coordinates in the C-arm imaging system, thereby determining the pixel position of the surgical instrument on the target medical image, thus allowing the surgical instrument to be displayed in the target medical image.

[0110] This application does not impose any restrictions on how surgical instruments are presented in the target medical image. For example, surgical instruments may be displayed in the target medical image in the form of dots, triangles, or patterns of surgical instruments.

[0111] In the embodiments of this application, on the one hand, the first conversion parameter is more accurate than the parameter obtained by conventional calibration. Surgical navigation based on the first conversion parameter can ensure the reliability of surgical navigation and help optimize the surgeon's surgical decision-making. On the other hand, the calibration process and the surgical navigation process can be integrated. When the patient is on the operating table, the calibration method provided in this application can be performed on the patient as the target object, and surgical navigation can be performed based on the first conversion parameter obtained by calibration, so as to achieve the effect of real-time calibration and fast surgical navigation.

[0112] In some embodiments of this application, the target medical image can be a first medical image and / or a second medical image. Since no surgical instruments are present in the first and second medical images, the original surgical instruments can be displayed on both images without removal, thus improving processing efficiency. Furthermore, by displaying medical images obtained from different shooting positions, doctors can easily obtain depth information about the surgical instruments.

[0113] In some other embodiments of this application, before displaying the target medical image acquired by the C-arm imaging system, the method may further include controlling the C-arm imaging system to re-capture a medical image as the target medical image. Specifically, if the physician is not satisfied with the clarity or shooting posture of the first and second medical images, new medical images may be re-captured for display. The new medical images may be one or more.

[0114] At this point, displaying the surgical instruments in the target medical image based on the real-time coordinates of the surgical instruments in the robotic arm coordinate system and the first transformation parameter may include: determining the second coordinate transformation matrix between the target medical image and the first medical image, and displaying the surgical instruments in the target medical image based on the real-time coordinates, the first transformation parameter, and the second coordinate transformation matrix.

[0115] Specifically, the process of obtaining the second coordinate transformation matrix between the target medical image and the first medical image can refer to the aforementioned process of obtaining the first coordinate transformation matrix. That is, given the shooting pose of the target medical image, the essential matrix is ​​obtained using the eight-point method, and then the second coordinate transformation matrix is ​​obtained through singular value decomposition based on the essential matrix. In other embodiments, if the image content of the target medical image and the first medical image is similar, the second coordinate transformation matrix can also be obtained by two-dimensional image registration.

[0116] The second coordinate transformation matrix can be used to perform coordinate transformation between the first medical image and the target medical image. Based on the real-time coordinates of the surgical instrument in the robotic arm coordinate system and the first transformation parameters, the pixel position of the surgical instrument in the first medical image can be determined. The second coordinate transformation matrix can convert the pixel position of the surgical instrument in the first medical image to the pixel position of the surgical instrument in the target medical image, thereby displaying the surgical instrument in the target medical image.

[0117] It should be noted that, for the sake of simplicity, the aforementioned method embodiments are described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders.

[0118] like Figure 9 The diagram shown is a structural schematic of a calibration device 900 provided in an embodiment of this application. The calibration device 900 is disposed on a medical device.

[0119] Specifically, the calibration device 900 may include:

[0120] The first image acquisition unit 901 is used to acquire the first medical image and the second medical image obtained by the C-arm imaging system capturing the target object in the first shooting pose and the second shooting pose, respectively.

[0121] The coordinate transformation matrix determination unit 902 is used to determine the first coordinate transformation matrix between the first shooting pose and the second shooting pose of the C-arm imaging system based on the first medical image and the second medical image.

[0122] The first projection matrix acquisition unit 903 is used to use the coordinate system of the C-arm imaging system corresponding to the first medical image as the world coordinate system to acquire the first projection matrix of the first medical image.

[0123] The second projection matrix acquisition unit 904 is used to obtain the second projection matrix of the second medical image based on the first projection matrix and the first coordinate transformation matrix.

[0124] The second image acquisition unit 905 is used to acquire the third medical image captured by the C-arm imaging system in the third shooting pose and the fourth medical image captured in the fourth shooting pose when the surgical instrument is located in the imaging field of the C-arm imaging system.

[0125] The world coordinate acquisition unit 906 is used to determine the first coordinates of the instrument feature points of the surgical instruments in the third medical image and the fourth medical image in the world coordinate system based on the first projection matrix and the second projection matrix.

[0126] The calibration unit 907 is used to determine the first transformation parameter between the robotic arm coordinate system and the C-arm imaging system coordinate system based on the first coordinate and the second coordinate of the instrument feature point in the robotic arm coordinate system.

[0127] In some embodiments of this application, the coordinate transformation matrix determination unit 902 is specifically used to: determine matching feature point pairs in the first medical image and the second medical image; and determine a first coordinate transformation matrix of the C-arm imaging system coordinate system of the second medical image relative to the C-arm imaging system coordinate system of the first medical image based on the pixel position of the feature point pairs.

[0128] In some embodiments of this application, the third shooting pose is the same as the first shooting pose, and the fourth shooting pose is the same as the second shooting pose. The world coordinate acquisition unit 906 is specifically used to: acquire at least three third feature points of the surgical instrument in the third medical image; acquire at least three fourth feature points of the surgical instrument in the fourth medical image corresponding to the third feature points; and obtain the first coordinates of the instrument feature points of the surgical instrument in the world coordinate system based on the first projection matrix, the second projection matrix, the third feature points, and the fourth feature points.

[0129] In some embodiments of this application, the world coordinate acquisition unit 906 is specifically used for: acquiring at least three third feature points of the surgical instrument in the third medical image; acquiring at least three fourth feature points of the surgical instrument in the fourth medical image corresponding to the third feature points; registering the third medical image with the first medical image to obtain a first registration matrix; registering the fourth medical image with the second medical image to obtain a second registration matrix; and obtaining the first coordinates of the instrument feature points of the surgical instrument in the world coordinate system based on the first registration matrix, the second registration matrix, the first projection matrix, the second projection matrix, the third feature points, and the fourth feature points.

[0130] In some embodiments of this application, the second shooting pose is obtained by controlling the C-arm imaging system to translate the first shooting pose along a preset direction by a preset distance.

[0131] In some embodiments of this application, the preset direction is the direction perpendicular to the sagittal plane of the target object.

[0132] It should be noted that, for the sake of convenience and brevity, the specific working process of the above-mentioned calibration device 900 can be found in the following reference: Figures 1 to 6 The corresponding process of the method will not be described in detail here.

[0133] like Figure 10 The diagram shown is a structural schematic of a calibration device 1000 provided in an embodiment of this application. The calibration device 1000 is disposed on a medical device.

[0134] Specifically, the calibration device 1000 may include:

[0135] The third image acquisition unit 1001 is used to acquire the fifth and sixth medical images obtained by the C-arm imaging system in the first and second shooting poses respectively when the surgical instruments are in the imaging field of the C-arm imaging system.

[0136] The coordinate transformation matrix determination unit 1002 is used to determine the first coordinate transformation matrix between the first shooting pose and the second shooting pose of the C-arm imaging system based on the fifth medical image and the sixth medical image;

[0137] The first projection matrix acquisition unit 1003 is used to use the coordinate system of the C-arm imaging system corresponding to the fifth medical image as the world coordinate system to acquire the first projection matrix of the fifth medical image.

[0138] The second projection matrix acquisition unit 1004 is used to obtain the second projection matrix of the sixth medical image based on the first projection matrix and the first coordinate transformation matrix.

[0139] The world coordinate acquisition unit 1005 is used to determine the first coordinates of the instrument feature points of the surgical instruments in the fifth medical image and the sixth medical image in the world coordinate system based on the first projection matrix and the second projection matrix.

[0140] The calibration unit 1006 is used to determine the first transformation parameter between the robotic arm coordinate system and the C-arm imaging system coordinate system based on the first coordinate and the second coordinate of the instrument feature point in the robotic arm coordinate system.

[0141] In some embodiments of this application, the coordinate transformation matrix determination unit 1002 may be specifically used to: determine matching feature point pairs in the fifth medical image and the sixth medical image; and determine a first coordinate transformation matrix of the C-arm imaging system coordinate system of the sixth medical image relative to the C-arm imaging system coordinate system of the fifth medical image based on the pixel position of the feature point pairs.

[0142] In some embodiments of this application, the world coordinate acquisition unit 1005 may be specifically used to: acquire at least three third feature points of the surgical instrument in the fifth medical image, acquire at least three fourth feature points of the surgical instrument in the sixth medical image corresponding to the third feature points; and obtain the first coordinates of the instrument feature points of the surgical instrument in the world coordinate system based on the first projection matrix, the second projection matrix, the third feature points, and the fourth feature points.

[0143] It should be noted that, for the sake of convenience and brevity, the specific working process of the above-mentioned calibration device 900 can be found in the following reference: Figure 7 The corresponding process of the method will not be described in detail here.

[0144] like Figure 11 The diagram shown is a structural schematic of a surgical navigation device 1100 provided in an embodiment of this application. The surgical navigation device 1100 is disposed on a medical device.

[0145] Specifically, the surgical navigation device 1100 may include:

[0146] Calibration unit 1101 is used to obtain the first conversion parameter;

[0147] Image display unit 1102 displays the target medical image captured by the C-arm imaging system;

[0148] The instrument display unit 1103 displays the surgical instrument in the target medical image based on the real-time coordinates of the surgical instrument in the robotic arm coordinate system and the first transformation parameter.

[0149] In some embodiments of this application, the target medical image is the first medical image and / or the second medical image.

[0150] In some embodiments of this application, the surgical navigation device 1100 further includes an image acquisition unit for controlling the C-arm imaging system to re-capture medical images as the target medical images; the instrument display unit 1103 is specifically used to: determine a second coordinate transformation matrix between the target medical image and the first medical image, and display the surgical instruments in the target medical image according to the real-time coordinates, the first transformation parameters and the second coordinate transformation matrix.

[0151] It should be noted that, for the sake of convenience and brevity, the specific working process of the surgical navigation device 1100 described above can be found in the following reference: Figure 8 The corresponding process of the method will not be described in detail here.

[0152] like Figure 12 The diagram shown is a schematic of a medical device 12 provided in an embodiment of this application. Specifically, the medical device 12 may include: a processor 120, a memory 121, and a computer program 122, such as a calibration program, stored in the memory 121 and executable on the processor 120.

[0153] When the processor 120 executes the computer program 122, it implements the steps in the above-described calibration method embodiments, for example... Figure 1 The steps S101 to S107 are shown. Alternatively, when the processor 120 executes the computer program 122, it implements the steps in the above-described calibration method embodiments, for example... Figure 7 Steps S701 to S706 are shown. Alternatively, when the processor 120 executes the computer program 122, it implements the steps in the various surgical navigation method embodiments described above, for example... Figure 8 Steps S801 to S803 are shown.

[0154] Alternatively, when the processor 120 executes the computer program 122, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 9 The functions of the first image acquisition unit 901, coordinate transformation matrix determination unit 902, first projection matrix acquisition unit 903, second projection matrix acquisition unit 904, second image acquisition unit 905, world coordinate acquisition unit 906, and calibration unit 907 shown are illustrated. Alternatively, when the processor 120 executes the computer program 122, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 10 The functions of the third image acquisition unit 1001, coordinate transformation matrix determination unit 1002, first projection matrix acquisition unit 1003, second projection matrix acquisition unit 1004, world coordinate acquisition unit 1005, and calibration unit 1006 shown are illustrated. Alternatively, when the processor 120 executes the computer program 122, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 11 The functions of the calibration unit 1101, image display unit 1102, and instrument display unit 1103 shown are illustrated.

[0155] The computer program can be divided into one or more modules / units, which are stored in the memory 121 and executed by the processor 120 to complete this application. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the medical device 12.

[0156] The medical device 12 may include, but is not limited to, a processor 120 and a memory 121. Those skilled in the art will understand that... Figure 12 This is merely an example of medical device 12 and does not constitute a limitation on medical device 12. It may include more or fewer components than shown, or combine certain components, or different components. For example, the medical device 12 may also include input / output devices, network access devices, buses, etc.

[0157] The processor 120 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0158] The memory 121 can be an internal storage unit of the medical device 12, such as a hard disk or memory of the medical device 12. The memory 121 can also be an external storage device of the medical device 12, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the medical device 12. Furthermore, the memory 121 can include both internal and external storage units of the medical device 12. The memory 121 is used to store the computer program and other programs and data required by the medical device 12. The memory 121 can also be used to temporarily store data that has been output or will be output.

[0159] It should be noted that, for the sake of convenience and brevity, the structure of the medical device 12 described above can also be referred to the specific description of the structure in the method embodiment, which will not be repeated here.

[0160] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0161] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0162] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for various specific applications, but such implementations should not be considered beyond the scope of this application.

[0163] In the embodiments provided in this application, it should be understood that the disclosed apparatus / devices and methods can be implemented in other ways. For example, the apparatus / device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0164] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0165] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0166] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0167] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A calibration method, characterized in that, include: The first and second medical images of the target object were obtained by the C-arm imaging system under the first and second shooting poses, respectively. Determine the first coordinate transformation matrix between the first shooting pose and the second shooting pose of the C-arm imaging system based on the first medical image and the second medical image; Using the coordinate system of the C-arm imaging system corresponding to the first medical image as the world coordinate system, the first projection matrix of the first medical image is obtained. The second projection matrix of the second medical image is obtained based on the first projection matrix and the first coordinate transformation matrix; When the surgical instruments are in the imaging field of the C-arm imaging system, acquire the third medical image captured by the C-arm imaging system in the third shooting pose and the fourth medical image captured in the fourth shooting pose. Based on the first projection matrix and the second projection matrix, determine the first coordinates of the instrument feature points of the surgical instruments in the third medical image and the fourth medical image in the world coordinate system. Based on the first coordinates and the second coordinates of the instrument feature points in the robotic arm's coordinate system, a first transformation parameter is determined between the robotic arm coordinate system and the C-arm imaging system coordinate system.

2. The calibration method as described in claim 1, characterized in that, Determining the first coordinate transformation matrix between the first and second imaging poses of the C-arm imaging system based on the first and second medical images includes: Identify matching feature point pairs in the first medical image and the second medical image; Based on the pixel positions of the feature point pairs, a first coordinate transformation matrix is ​​determined relative to the C-arm imaging system coordinate system of the second medical image and the first medical image.

3. The calibration method as described in claim 1, characterized in that, The third shooting pose is the same as the first shooting pose, and the fourth shooting pose is the same as the second shooting pose; Based on the first projection matrix and the second projection matrix, determine the first coordinates of the instrument feature points of the surgical instruments in the third and fourth medical images in the world coordinate system, including: Acquire at least three third feature points of the surgical instrument in the third medical image, and acquire at least three fourth feature points of the surgical instrument in the fourth medical image corresponding to the third feature points; Based on the first projection matrix, the second projection matrix, the third feature point, and the fourth feature point, the first coordinates of the instrument feature point of the surgical instrument in the world coordinate system are obtained.

4. The calibration method as described in claim 1, characterized in that, Based on the first projection matrix and the second projection matrix, determine the first coordinates of the instrument feature points of the surgical instruments in the third and fourth medical images in the world coordinate system, including: Acquire at least three third feature points of the surgical instrument in the third medical image, and acquire at least three fourth feature points of the surgical instrument in the fourth medical image corresponding to the third feature points; The third medical image is registered with the first medical image to obtain a first registration matrix; The fourth medical image and the second medical image are registered to obtain a second registration matrix; Based on the first registration matrix, the second registration matrix, the first projection matrix, the second projection matrix, the third feature point, and the fourth feature point, the first coordinate of the instrument feature point of the surgical instrument in the world coordinate system is obtained.

5. The calibration method according to any one of claims 1-4, characterized in that, The second shooting pose is obtained by controlling the C-arm imaging system to translate a preset distance from the first shooting pose along a preset direction.

6. The calibration method as described in claim 5, characterized in that, The preset direction is the direction perpendicular to the sagittal plane of the target object.

7. A calibration method, characterized in that, include: When the surgical instruments are in the imaging field of the C-arm imaging system, the fifth and sixth medical images are obtained by the C-arm imaging system taking pictures of the target object in the first and second shooting poses, respectively. Based on the fifth medical image and the sixth medical image, determine the first coordinate transformation matrix between the first shooting pose and the second shooting pose of the C-arm imaging system; Using the coordinate system of the C-arm imaging system corresponding to the fifth medical image as the world coordinate system, the first projection matrix of the fifth medical image is obtained. The second projection matrix of the sixth medical image is obtained based on the first projection matrix and the first coordinate transformation matrix; Based on the first projection matrix and the second projection matrix, determine the first coordinates of the instrument feature points of the surgical instruments in the fifth medical image and the sixth medical image in the world coordinate system. Based on the first coordinates and the second coordinates of the instrument feature points in the robotic arm's coordinate system, a first transformation parameter is determined between the robotic arm coordinate system and the C-arm imaging system coordinate system.

8. The calibration method as described in claim 7, characterized in that, Determining the first coordinate transformation matrix between the first and second imaging poses of the C-arm imaging system based on the fifth and sixth medical images includes: In the fifth and sixth medical images, matching feature point pairs are identified; Based on the pixel positions of the feature point pairs, a first coordinate transformation matrix is ​​determined relative to the C-arm imaging system coordinate system of the sixth medical image and the C-arm imaging system coordinate system of the fifth medical image.

9. The calibration method as described in claim 7, characterized in that, Based on the first projection matrix and the second projection matrix, the first coordinates of the instrument feature points of the surgical instruments in the fifth and sixth medical images in the world coordinate system are determined, including: Acquire at least three third feature points of the surgical instrument in the fifth medical image, and acquire at least three fourth feature points of the surgical instrument in the sixth medical image corresponding to the third feature points; Based on the first projection matrix, the second projection matrix, the third feature point, and the fourth feature point, the first coordinates of the instrument feature point of the surgical instrument in the world coordinate system are obtained.

10. A surgical navigation method, characterized in that, include: Obtain a first conversion parameter, wherein the first conversion parameter is obtained by the calibration method according to any one of claims 1-6 or the calibration method according to any one of claims 7-9; Displaying the target medical image captured by the C-arm imaging system; The surgical instruments are displayed in the target medical image based on their real-time coordinates in the robotic arm coordinate system and the first transformation parameter.

11. The surgical navigation method as described in claim 10, characterized in that, The target medical image is a first medical image and / or a second medical image.

12. The surgical navigation method as described in claim 10, characterized in that, Before displaying the target medical image captured by the C-arm imaging system, the surgical navigation method further includes: controlling the C-arm imaging system to re-capture the medical image as the target medical image; The step of displaying the surgical instrument in the target medical image based on the real-time coordinates of the surgical instrument in the robotic arm coordinate system and the first transformation parameter includes: determining a second coordinate transformation matrix between the target medical image and the first medical image, and displaying the surgical instrument in the target medical image based on the real-time coordinates, the first transformation parameter, and the second coordinate transformation matrix.

13. A medical device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the calibration method as described in any one of claims 1 to 6; or, when the processor executes the computer program, it implements the steps of the calibration method as described in any one of claims 7 to 9; or, when the processor executes the computer program, it implements the steps of the surgical navigation method as described in any one of claims 10 to 12.

14. A computer program product, characterized in that, The method includes a computer program that, when executed, causes the calibration method as described in any one of claims 1 to 6 to be performed; or, when executed, causes the calibration method as described in any one of claims 7 to 9 to be performed; or, when executed, causes the surgical navigation method as described in any one of claims 10 to 12 to be performed.

Images