Calibration method, surgical navigation method, medical device and computer program product

Abstract

The invention is suitable for the technical field of medical treatment, and provides a calibration method, a surgical navigation method, medical equipment and a computer program product. The calibration method comprises the following steps: acquiring a first medical image and a second medical image shot by a C-shaped arm imaging system under two shooting poses; when the surgical instrument is located in the imaging view field, a third medical image and a fourth medical image shot by the C-shaped arm imaging system under the two shooting poses are obtained; a C-shaped arm imaging system coordinate system corresponding to the first medical image serves as a world coordinate system, and first coordinates of instrument feature points of the surgical instruments in the third medical image and the fourth medical image in the world coordinate system are determined; and according to the first coordinate and a second coordinate of the instrument feature point in a mechanical arm coordinate system of the mechanical arm, a first conversion parameter between the mechanical arm coordinate system and a C-shaped arm imaging system coordinate system is determined. According to the embodiment of the invention, calibration can be completed in real time without a calibration target, so that conversion parameters are more accurate.

Description

Technical Field

[0001] The present application belongs to the field of medical technology, and in particular relates to a calibration method, a surgical navigation method, a medical device and a computer program product. Background Art

[0002] When using a C-arm for surgical navigation, it is usually necessary to calibrate the C-arm imaging system. The calibration of the C-arm imaging system can be roughly regarded as the calibration of the camera, which usually requires the use of a calibration target to obtain the conversion parameters between coordinate systems through online calibration (calibration by photographing the calibration target on site) or offline calibration (calibration with a large number of calibration targets before use, and acquisition through table interpolation when using). On the one hand, the calibration target requires special design, processing and measurement. When using the calibration target, there are certain constraints, which will increase the use time and difficulty of use. On the other hand, the conversion parameters obtained by interpolation during offline calibration often have errors. Summary of the invention

[0003] The embodiments of the present application provide a calibration method, a surgical navigation method, a medical device and a computer program product, which can complete the calibration in real time without being separated from the calibration target, so that the conversion parameters are more accurate.

[0004] A first aspect of an embodiment of the present application provides a calibration method, including: obtaining a first medical image and a second medical image obtained by respectively photographing a target object by a C-arm imaging system in a first shooting posture and a second shooting posture; determining a first coordinate conversion matrix between the first shooting posture and the second shooting posture of the C-arm imaging system according to the first medical image and the second medical image; taking the C-arm imaging system coordinate system corresponding to the first medical image as the world coordinate system, and obtaining a first projection matrix of the first medical image; obtaining a second projection matrix of the second medical image according to the first projection matrix and the first coordinate conversion matrix; when a surgical instrument is located in the imaging field of view of the C-arm imaging system, obtaining a third medical image photographed by the C-arm imaging system in a third shooting posture and a fourth medical image photographed in a fourth shooting posture; determining a first coordinate of an instrument feature point of the surgical instrument in the third medical image and the fourth medical image in the world coordinate system according to the first projection matrix and the second projection matrix; determining a first conversion parameter between the robotic arm coordinate system and the C-arm imaging system coordinate system according to the first coordinate and the second coordinate of the instrument feature point in the robotic arm coordinate system of the robotic arm.

[0005] In some embodiments of the first aspect, determining a first coordinate transformation matrix between the first shooting posture and the second shooting posture of the C-arm imaging system based on the first medical image and the second medical image includes: determining matching feature point pairs in the first medical image and the second medical image; and determining, based on pixel positions of the feature point pairs, a first coordinate transformation matrix of the C-arm imaging system coordinate system of the second medical image relative to the C-arm imaging system coordinate system of the first medical image.

[0006] In some embodiments of the first aspect, the third shooting posture is the same as the first shooting posture, and the fourth shooting posture is the same as the second shooting posture; determining the first coordinates of the instrument feature points of the surgical instrument in the third medical image and the fourth medical image in the world coordinate system according to the first projection matrix and the second projection matrix, including: obtaining at least three third feature points of the surgical instrument in the third medical image, and obtaining at least three fourth feature points of the surgical instrument in the fourth medical image corresponding to the third feature points; obtaining the first coordinates of the instrument feature points of the surgical instrument in the world coordinate system according to the first projection matrix, the second projection matrix, the third feature points and the fourth feature points.

[0007] In some embodiments of the first aspect, determining the first coordinates of the instrument feature points of the surgical instrument in the third medical image and the fourth medical image in the world coordinate system according to the first projection matrix and the second projection matrix, includes: obtaining at least three third feature points of the surgical instrument in the third medical image, and obtaining at least three fourth feature points of the surgical instrument in the fourth medical image corresponding to the third feature points; registering the third medical image with the first medical image to obtain a first registration matrix; registering the fourth medical image with the second medical image to obtain a second registration matrix; and obtaining the first coordinates of the instrument feature points of the surgical instrument in the world coordinate system according to the first registration matrix, the second registration matrix, the first projection matrix, the second projection matrix, the third feature points and the fourth feature points.

[0008] In some embodiments of the first aspect, the second shooting posture is obtained by controlling the C-arm imaging system to translate a preset distance along a preset direction from the first shooting posture.

[0009] In some embodiments of the first aspect, the preset direction is a direction perpendicular to the sagittal plane of the target object.

[0010] A second aspect of an embodiment of the present application provides a calibration method, including: when a surgical instrument is located in the imaging field of view of the C-arm imaging system, obtaining a fifth medical image and a sixth medical image obtained by respectively photographing the target object by the C-arm imaging system in a first shooting posture and a second shooting posture; determining a first coordinate conversion matrix between the first shooting posture and the second shooting posture of the C-arm imaging system according to the fifth medical image and the sixth medical image; taking the C-arm imaging system coordinate system corresponding to the fifth medical image as the world coordinate system, obtaining a first projection matrix of the fifth medical image; obtaining a second projection matrix of the sixth medical image according to the first projection matrix and the first coordinate conversion matrix; determining a first coordinate of an instrument feature point of the surgical instrument in the fifth medical image and the sixth medical image in the world coordinate system according to the first projection matrix and the second projection matrix; determining a first conversion parameter between the robotic arm coordinate system and the C-arm imaging system coordinate system according to the first coordinate and the second coordinate of the instrument feature point in the robotic arm coordinate system of the robotic arm.

[0011] In some embodiments of the second aspect, determining a first coordinate transformation matrix between the first shooting posture and the second shooting posture of the C-arm imaging system according to the fifth medical image and the sixth medical image includes: determining matching feature point pairs in the fifth medical image and the sixth medical image; and determining, according to the pixel positions of the feature point pairs, a first coordinate transformation matrix of the C-arm imaging system coordinate system of the sixth medical image relative to the C-arm imaging system coordinate system of the fifth medical image.

[0012] In some embodiments of the second aspect, determining the first coordinates of the instrument feature points of the surgical instrument in the fifth medical image and the sixth medical image in the world coordinate system based on the first projection matrix and the second projection matrix includes: obtaining at least three third feature points of the surgical instrument in the fifth medical image, and obtaining at least three fourth feature points of the surgical instrument in the sixth medical image corresponding to the third feature points; and obtaining the first coordinates of the instrument feature points of the surgical instrument in the world coordinate system based on the first projection matrix, the second projection matrix, the third feature points and the fourth feature points.

[0013] A third aspect of an embodiment of the present application provides a surgical navigation method, comprising: acquiring a first conversion parameter, wherein the first conversion parameter is obtained according to the calibration method described in any one of the first aspect or the calibration method described in any one of the second aspect; displaying a target medical image captured by a C-arm imaging system; and displaying the surgical instrument in the target medical image based on the real-time coordinates of the surgical instrument in the robotic arm coordinate system and the first conversion parameter.

[0014] In some embodiments of the third aspect, the target medical image is the first medical image and / or the second medical image.

[0015] In some embodiments of the third aspect, before displaying the target medical image captured by the C-arm imaging system, it also includes: controlling the C-arm imaging system to re-record the medical image as the target medical image; displaying the surgical instrument in the target medical image based on the real-time coordinates of the surgical instrument in the robotic arm coordinate system and the first conversion parameters, including: determining a second coordinate conversion matrix between the target medical image and the first medical image, and displaying the surgical instrument in the target medical image based on the real-time coordinates, the first conversion parameters and the second coordinate conversion matrix.

[0016] A calibration device provided in a fourth aspect of an embodiment of the present application includes: a first image acquisition unit, used to acquire a first medical image and a second medical image obtained by respectively photographing a target object by a C-arm imaging system in a first shooting posture and a second shooting posture; a coordinate transformation matrix determination unit, used to determine a first coordinate transformation matrix between the first shooting posture and the second shooting posture of the C-arm imaging system according to the first medical image and the second medical image; a first projection matrix acquisition unit, used to use the C-arm imaging system coordinate system corresponding to the first medical image as the world coordinate system to acquire a first projection matrix of the first medical image; a second projection matrix acquisition unit, used to obtain a first projection matrix of the first medical image according to the first projection matrix and the first coordinate transformation matrix a second image acquisition unit, for acquiring, when the surgical instrument is located in the imaging field of the C-arm imaging system, a third medical image captured by the C-arm imaging system in a third shooting posture and a fourth medical image captured in a fourth shooting posture; a world coordinate acquisition unit, for determining, according to the first projection matrix and the second projection matrix, first coordinates of instrument feature points of the surgical instrument in the third medical image and the fourth medical image in the world coordinate system; a calibration unit, for determining, according to the first coordinates and the second coordinates of the instrument feature points in the robotic arm coordinate system of the robotic arm, first conversion parameters between the robotic arm coordinate system and the C-arm imaging system coordinate system.

[0017] A calibration device provided in a fifth aspect of an embodiment of the present application includes: a third image acquisition unit, used to acquire a fifth medical image and a sixth medical image obtained by respectively photographing a target object by the C-arm imaging system in a first shooting posture and a second shooting posture when the surgical instrument is located in the imaging field of view of the C-arm imaging system; a coordinate conversion matrix determination unit, used to determine a first coordinate conversion matrix between the first shooting posture and the second shooting posture of the C-arm imaging system according to the fifth medical image and the sixth medical image; a first projection matrix acquisition unit, used to use the C-arm imaging system coordinate system corresponding to the fifth medical image as a world coordinate system to acquire a first projection matrix of the fifth medical image; a second projection matrix acquisition unit, used to obtain a second projection matrix of the sixth medical image according to the first projection matrix and the first coordinate conversion matrix; a world coordinate acquisition unit, used to determine the first coordinates of the instrument feature points of the surgical instrument in the fifth medical image and the sixth medical image in the world coordinate system according to the first projection matrix and the second projection matrix; a calibration unit, used to determine the first conversion parameter between the robot coordinate system and the C-arm imaging system coordinate system according to the first coordinate and the second coordinate of the instrument feature point in the robot coordinate system of the robot.

[0018] A surgical navigation device provided in a sixth aspect of an embodiment of the present application includes: a calibration unit for acquiring a first conversion parameter, wherein the first conversion parameter is obtained according to the calibration method described in any one of the first aspects; an image display unit for displaying a target medical image taken by a C-arm imaging system; and an instrument display unit for displaying the surgical instrument in the target medical image according to the real-time coordinates of the surgical instrument in the robotic arm coordinate system and the first conversion parameter.

[0019] A seventh aspect of an embodiment of the present application provides a medical device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the calibration method described in any one of the first aspect when executing the computer program, or implements the steps of the calibration method described in any one of the second aspect when executing the computer program, or implements the steps of the surgical navigation method described in any one of the third aspect when executing the computer program.

[0020] An eighth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the calibration method described in the first aspect are implemented. Alternatively, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the calibration method described in the second aspect are implemented. Alternatively, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the surgical navigation method described in the third aspect are implemented.

[0021] A ninth aspect of an embodiment of the present application provides a computer program product, comprising a computer program, wherein when the computer program is run, the calibration method described in the first aspect is executed, or, when the computer program is run, the calibration method described in the second aspect is executed, or, when the computer program is run, the surgical navigation method described in the third aspect is executed.

[0022] In an embodiment of the present application, a first medical image and a second medical image are obtained by respectively acquiring a C-arm imaging system to photograph a target object at a first shooting posture and a second shooting posture to determine a first coordinate conversion matrix between the first shooting posture and the second shooting posture, and a C-arm imaging system coordinate system corresponding to the first medical image is used as a world coordinate system to obtain a first projection matrix and a second projection matrix of the first medical image, and when the surgical instrument is located in the imaging field of view of the C-arm imaging system, a third medical image photographed at a third shooting posture and a fourth medical image photographed at a fourth shooting posture are obtained, and then, according to the first projection matrix and the second projection matrix, the first coordinates of the instrument feature points of the surgical instrument in the third medical image and the fourth medical image in the world coordinate system are determined, and according to the first coordinates and the second coordinates of the instrument feature points in the robotic arm coordinate system of the robotic arm, the first conversion parameters between the robotic arm coordinate system and the C-arm imaging system coordinate system are determined, and the calibration can be completed in real time during the process of shooting medical images without being detached from the calibration target. Compared with the conversion parameters obtained by interpolation, this method can obtain more accurate conversion parameters based on the on-site installation of the C-arm imaging system. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0024] Figure 1 It is a schematic diagram of an implementation flow of a calibration method provided in an embodiment of the present application;

[0025] Figure 2 is a schematic diagram of shooting a first medical image provided by an embodiment of the present application;

[0026] Figure 3 is a schematic diagram of capturing a second medical image provided by an embodiment of the present application;

[0027] Figure 4 is a schematic diagram of a specific implementation process of determining a first coordinate transformation matrix provided in an embodiment of the present application;

[0028] Figure 5 A schematic diagram of taking a third medical image provided by an embodiment of the present application;

[0029] Figure 6 A schematic diagram of taking a fourth medical image provided by an embodiment of the present application;

[0030] Figure 7 It is a schematic diagram of the implementation flow of another calibration method provided in an embodiment of the present application;

[0031] Figure 8 It is a schematic diagram of the implementation flow of the surgical navigation method provided in the embodiment of the present application;

[0032] Fig. 9 is a structural schematic diagram of a calibration device provided in an embodiment of the present application;

[0033] Fig.10 is a structural schematic diagram of another calibration device provided in an embodiment of the present application;

[0034] Fig.11 is a structural schematic diagram of a surgical navigation device provided in an embodiment of the present application;

[0035] Fig.12 It is a schematic diagram of the structure of the medical device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work are protected by the present application.

[0037] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0038] In the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0039] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0040] When using a C-arm for surgical navigation, it is usually necessary to calibrate the C-arm imaging system. The calibration of the C-arm imaging system can be roughly regarded as the calibration of the camera, which usually requires the use of a calibration target to obtain the conversion parameters between coordinate systems through online calibration (calibration by photographing the calibration target on site) or offline calibration (calibration with a large number of calibration targets before use, and acquisition through table interpolation when using). On the one hand, the calibration target requires special design, processing and measurement. When using the calibration target, there are certain constraints, which will increase the use time and difficulty of use. On the other hand, the conversion parameters obtained by interpolation during offline calibration often have errors.

[0041] In view of this, the present application proposes a calibration method, which can complete the calibration in real time without the calibration target, so that the conversion parameters are more accurate.

[0042] In order to illustrate the technical solution of the present application, a specific embodiment is provided below for illustration.

[0043] Please refer to Figure 1 , Figure 1 A schematic diagram of the implementation process of a calibration method provided in an embodiment of the present application is shown. The method can be applied to a medical device, which can be a computer, a server or other intelligent device, and the present application does not impose any limitation on this.

[0044] In some embodiments of the present application, the above-mentioned medical device can be connected to a C-arm imaging system and a robotic arm. The C-arm imaging system can generate X-rays through an X-ray tube on a C-frame, and a receiving end receives the X-rays to obtain a medical image. The medical image can refer to an X-ray image (X-ray). The end of the robotic arm can be provided with a surgical instrument for performing a surgical operation.

[0045] Specifically, the above calibration method may include the following steps S101 to S107.

[0046] Step S101 : acquiring a first medical image and a second medical image obtained by photographing a target object by a C-arm imaging system in a first photographing posture and a second photographing posture, respectively.

[0047] The first shooting posture and the second shooting posture are different. By controlling the C-arm imaging system to shoot the target object in the first shooting posture and the second shooting posture respectively, two medical images can be obtained, and the two medical images are the first medical image and the second medical image. The first medical image and the second medical image can correspond to a shooting posture respectively.

[0048] In the implementation of the present application, the target object may be a person, an animal or other photographed object, and the present application does not limit this. In some implementations, the target object may be a patient, and during the surgical navigation process of the patient, the first medical image and the second medical image may be obtained by photographing the patient in the first shooting posture and the second shooting posture respectively through the C-arm imaging system, so that the calibration process can be integrated into the surgical navigation process.

[0049] In some embodiments of the present application, there are matching object feature points between the first medical image and the second medical image. Object feature points are feature points on the target object. These object feature points can be feature points located at the target position (for example, on a block, edge, or corner point), and the number is less than a number threshold (i.e., relatively rare, much less than the number of pixels) for subsequent identification. For example, object feature points may include, but are not limited to: the upper and lower end plates and transverse processes of the spinous processes of the vertebral body of the spine, the long axis and short axis of the obturator foramen of the pelvis, and the lesser trochanter of the femur.

[0050] It should be noted that the present application does not restrict the order in which the first medical image and the second medical image are taken. That is to say, when the C-arm imaging system is shooting in the first shooting position, the obtained medical image can be the first medical image or the second medical image.

[0051] Step S102: determining a first coordinate transformation matrix between a first shooting posture and a second shooting posture of a C-arm imaging system according to the first medical image and the second medical image.

[0052] Specifically, since the first shooting posture and the second shooting posture are known, the posture change between the first shooting posture and the second shooting posture can be inferred, and based on the matching point pairs in the first medical image and the second medical image, the first coordinate transformation matrix between the first shooting posture and the second shooting posture can be calculated. The first coordinate transformation matrix can be used to perform coordinate transformation between the C-arm imaging system coordinate system corresponding to the first medical image and the C-arm imaging system coordinate system corresponding to the second medical image, and can be composed of a rotation matrix R and a translation matrix t.

[0053] Step S103: Using the C-arm imaging system coordinate system corresponding to the first medical image as the world coordinate system, and acquiring a first projection matrix of the first medical image.

[0054] In an embodiment of the present application, the projection matrix is ​​the product of an internal parameter matrix and an external parameter matrix. Among them, the internal parameter matrix of the C-arm imaging system is related to the structure of the C-arm itself, and the internal parameter matrix is ​​a priori information. Since the coordinate system of the C-arm imaging system corresponding to the first medical image is used as the world coordinate system, the rotation matrix R in the external parameter matrix corresponding to the first medical image is a 3×3 unit matrix, and the translation matrix t is a 3×1 zero matrix. By multiplying the internal parameter matrix and the external parameter matrix corresponding to the first medical image, the first projection matrix of the first medical image can be obtained.

[0055] Step S104 , obtaining a second projection matrix of the second medical image according to the first projection matrix and the first coordinate transformation matrix.

[0056] In an embodiment of the present application, the first projection matrix can be transformed based on the first coordinate transformation matrix between the first shooting posture and the second shooting posture to obtain a second projection matrix of the second medical image. Specifically, when the C-arm imaging system coordinate system corresponding to the first medical image is used as the world coordinate system, the external parameter matrix corresponding to the second medical image is the first coordinate transformation matrix, and the internal parameter matrix and the external parameter matrix corresponding to the second medical image are multiplied to obtain the second projection matrix of the second medical image.

[0057] Step S105, when the surgical instrument is located in the imaging field of the C-arm imaging system, a third medical image captured by the C-arm imaging system in a third shooting posture and a fourth medical image captured in a fourth shooting posture are acquired.

[0058] In an embodiment of the present application, by controlling the robotic arm, the surgical instrument at the end of the robotic arm can be moved into the imaging field of view of the C-arm imaging system. Preferably, the surgical instrument can be moved near the patient area so that the aforementioned object feature points are not blocked in the imaging field of view. Subsequently, the third medical image and the fourth medical image can be obtained by shooting at the third shooting posture and the fourth shooting posture respectively through the C-arm imaging system.

[0059] The third medical image and the fourth medical image correspond to the first medical image and the second medical image, respectively. Specifically, the third medical image can be shot in a posture that is the same as or similar to the first shooting posture, and the fourth medical image can be shot in a posture that is the same as or similar to the second shooting posture.

[0060] Step S106: determining first coordinates of instrument feature points of the surgical instruments in the third medical image and the fourth medical image in the world coordinate system according to the first projection matrix and the second projection matrix.

[0061] The instrument feature point refers to a feature point on the surgical instrument. The C-arm imaging system coordinate system corresponding to the first medical image is used as the world coordinate system, and the first coordinate of the instrument feature point on the surgical instrument in the world coordinate system can be calculated according to the first coordinate conversion matrix, the first projection matrix, and the second projection matrix.

[0062] Step S107 , determining a first conversion parameter between the robotic arm coordinate system and the C-arm imaging system coordinate system according to the first coordinate and the second coordinate of the instrument feature point in the robotic arm coordinate system.

[0063] In some embodiments of the present application, since the surgical instrument is moved by the movement of the robotic arm, the position of the surgical instrument in the robotic arm coordinate system of the robotic arm is known, and the instrument feature point is located on the surgical instrument, so the second coordinate of the instrument feature point in the robotic arm coordinate system of the robotic arm is a priori information. According to the first coordinate of the instrument feature point in the world coordinate system and the second coordinate of the instrument feature point in the robotic arm coordinate system of the robotic arm, the conversion relationship between the world coordinate system and the robotic arm coordinate system can be determined. Since the world coordinate system is the C-arm imaging system coordinate system corresponding to the first medical image, the first conversion parameter between the robotic arm coordinate system and the C-arm imaging system coordinate system can be obtained. The first conversion parameter can be used to convert the coordinates of the robotic arm coordinate system into the coordinates of the C-arm imaging system coordinate system, or to convert the coordinates of the C-arm imaging system coordinate system into the coordinates of the robotic arm coordinate system.

[0064] It should be noted that the above-mentioned robot arm coordinate system can be the base coordinate system of the robot arm (that is, the coordinate system established with the installation base of the robot arm as the reference), or it can be a coordinate system established with other parts of the robot arm as the reference, and this application does not impose any restrictions on this.

[0065] In an embodiment of the present application, a first medical image and a second medical image are obtained by respectively acquiring a C-arm imaging system to photograph a target object at a first shooting posture and a second shooting posture to determine a first coordinate conversion matrix between the first shooting posture and the second shooting posture, and a C-arm imaging system coordinate system corresponding to the first medical image is used as a world coordinate system to obtain a first projection matrix and a second projection matrix of the first medical image, and when the surgical instrument is located in the imaging field of view of the C-arm imaging system, a third medical image photographed at a third shooting posture and a fourth medical image photographed at a fourth shooting posture are obtained, and then, according to the first projection matrix and the second projection matrix, the first coordinates of the instrument feature points of the surgical instrument in the third medical image and the fourth medical image in the world coordinate system are determined, and according to the first coordinates and the second coordinates of the instrument feature points in the robotic arm coordinate system of the robotic arm, the first conversion parameters between the robotic arm coordinate system and the C-arm imaging system coordinate system are determined, and the calibration can be completed in real time during the process of shooting medical images without being detached from the calibration target. Compared with the conversion parameters obtained by interpolation, this method can obtain more accurate conversion parameters based on the on-site installation of the C-arm imaging system.

[0066] In some embodiments of the present application, the second shooting posture is obtained by controlling the C-arm imaging system to translate the first shooting posture along a preset direction and a preset distance.

[0067] For details, please refer to Figure 2 and Figure 3 , the C-arm imaging system can be controlled to capture the area with obvious object feature points in the first first shooting posture to obtain the first medical image. When the target object is stationary, the C-arm imaging system is translated along the preset direction and the preset distance to ensure that the object feature points in the first shooting posture can also appear in the imaging field of view, and the area with obvious object feature points is captured to obtain the second medical image.

[0068] In some embodiments of the present application, the preset direction may be the horizontal direction of the C-arm imaging system. Specifically, when the C-arm imaging system is used to scan the coronal plane of the target object, the preset direction is the vertical direction of the sagittal plane of the target object. In this way, by moving a smaller preset distance, the requirement that the feature points of the object appear in the imaging field of view can be met, and the two medical images can be different, that is, the pixel positions of the same feature points of the object in the two medical images are different.

[0069] In some embodiments of the present application, Figure 4 As shown, determining a first coordinate transformation matrix between a first shooting posture and a second shooting posture of a C-arm imaging system according to a first medical image and a second medical image may include steps S401 to S402.

[0070] Step S401: determining matching feature point pairs in a first medical image and a second medical image.

[0071] Specifically, feature points can be extracted from the first medical image and the second medical image respectively, and the feature points on the two medical images are matched to obtain matching feature point pairs. Each feature point pair includes a first feature point on the first medical image and a second feature point on the second medical image corresponding to the first feature point. The first feature point and the second feature point can be object feature points on the target object. The number of feature point pairs can be at least eight pairs.

[0072] Step S402: determining a first coordinate transformation matrix of the C-arm imaging system coordinate system of the second medical image relative to the C-arm imaging system coordinate system of the first medical image according to the pixel positions of the feature point pairs.

[0073] Specifically, according to the pixel position of the first feature point on the first medical image and the pixel position of the second feature point on the second medical image in the feature point pair, the eight-point method can be used to calculate the essential matrix E. The essential matrix E is used to describe the relationship between the coordinates of a point in space in the camera coordinate system under different shooting postures. According to the essential matrix E, the first coordinate transformation matrix of the C-arm imaging system coordinate system of the second medical image relative to the C-arm imaging system coordinate system of the first medical image can be calculated by singular value decomposition, and the first coordinate transformation matrix can be expressed as a rotation matrix R and a translation matrix t.

[0074] In some embodiments of the present application, in the process of acquiring the third medical image and the fourth medical image captured by the C-arm imaging system, the third shooting posture may be the same as the first shooting posture, and the fourth shooting posture may be the same as the second shooting posture.

[0075] Specifically, Figure 5 and Figure 6 As shown, when the C-arm imaging system can record the first shooting position and the second shooting position, or can be accurately moved to the first shooting position and the second shooting position, the C-arm imaging system can be controlled to shoot at the first shooting position and the second shooting position to obtain a third medical image and a fourth medical image, respectively.

[0076] At this time, the third medical image and the first medical image are taken at the same shooting posture, and the fourth medical image and the second medical image are taken at the same shooting posture. Determining the first coordinates of the instrument feature points of the surgical instrument in the third medical image and the fourth medical image in the world coordinate system according to the first projection matrix and the second projection matrix may include: obtaining at least three third feature points of the surgical instrument in the third medical image, obtaining at least three fourth feature points of the surgical instrument in the fourth medical image corresponding to the third feature points; obtaining the first coordinates of the instrument feature points of the surgical instrument in the world coordinate system according to the first projection matrix, the second projection matrix, the third feature points and the fourth feature points.

[0077] Among them, the third feature point and the fourth feature point are both instrument feature points of the surgical instrument. Among the at least three third feature points, there are at least three non-collinear third feature points. Among the at least three fourth feature points, there are at least three non-collinear fourth feature points. At least three pairs of mutually matching feature point pairs can be formed between the at least three third feature points and the at least three fourth feature points.

[0078] Since the third shooting posture is the same as the first shooting posture, and the fourth shooting posture is the same as the second shooting posture, the pixel position of the third feature point in the third medical image can be directly used as the pixel position of the third feature point in the first medical image; the pixel position of the fourth feature point in the fourth medical image can be directly used as the pixel position of the fourth feature point in the second medical image. In other words, the first projection matrix of the first medical image is the projection matrix of the third medical image; the first projection matrix of the second medical image is the projection matrix of the fourth medical image. At this time, when the first projection matrix of the first medical image and the second projection matrix of the second medical image are both known, the pixel position of the third feature point in the third medical image and the pixel position of the fourth feature point in the fourth medical image can be used to jointly solve the first coordinate of the instrument feature point of the surgical instrument in the world coordinate system.

[0079] This method can save the process of registering the third medical image with the first medical image and registering the fourth medical image with the second medical image, and has high computational efficiency, but has high requirements on the accuracy of the shooting posture.

[0080] In some other embodiments of the present application, after obtaining the third medical image and the fourth medical image, the method may further include: registering the third medical image with the first medical image; and registering the fourth medical image with the second medical image.

[0081] Specifically, determining the first coordinates of the instrument feature points of the surgical instrument in the third medical image and the fourth medical image in the world coordinate system according to the first projection matrix and the second projection matrix may include: obtaining at least three third feature points of the surgical instrument in the third medical image, and obtaining at least three fourth feature points of the surgical instrument in the fourth medical image corresponding to the third feature points. The third medical image is registered with the first medical image to obtain a first registration matrix. The fourth medical image is registered with the second medical image to obtain a second registration matrix. According to the first registration matrix, the second registration matrix, the first projection matrix, the second projection matrix, the third feature point and the fourth feature point, the first coordinates of the instrument feature point of the surgical instrument in the world coordinate system are obtained.

[0082] When the C-arm imaging system is manually controlled, the C-arm imaging system can be controlled to shoot in a third shooting posture that is similar to or the same as the first shooting posture, and to shoot in a fourth shooting posture that is similar to or the same as the second shooting posture. And through image registration, the third medical image and the fourth medical image are aligned to the first shooting posture and the second shooting posture. For the third medical image corresponding to the first medical image, it can be registered with the first medical image. For the fourth medical image corresponding to the second medical image, it can be registered with the second medical image. In this way, after the registration is completed, the first registration matrix can be used to convert the pixel position of the instrument feature point of the surgical instrument in the third medical image to the pixel position in the first medical image; the second registration matrix can be used to convert the pixel position of the instrument feature point of the surgical instrument in the fourth medical image to the pixel position in the second medical image.

[0083] Using the first registration matrix, the pixel position of the third feature point in the third medical image can be converted to the pixel position of the third feature point in the first medical image. Using the second registration matrix, the pixel position of the fourth feature point in the fourth medical image can be converted to the pixel position of the fourth feature point in the second medical image. At this time, when the first projection matrix of the first medical image and the second projection matrix of the second medical image are both known, the pixel position of the third feature point in the first medical image and the pixel position of the fourth feature point in the second medical image can be used to jointly solve the first coordinate of the instrument feature point of the surgical instrument in the world coordinate system.

[0084] After the first coordinate is obtained, a first conversion parameter between the manipulator coordinate system and the C-arm imaging system coordinate system may be determined according to the first coordinate and a second coordinate of the instrument feature point in the manipulator coordinate system of the manipulator.

[0085] In the implementation of the present application, by shooting four medical images, the calibration can be completed in real time without the calibration target. This method does not require the collection of a large amount of calibration target data under different shooting postures before use, nor does it require the conversion parameters obtained by interpolation. More accurate conversion parameters can be obtained based on the on-site installation conditions and the structural dimensions of the C-arm imaging system. In addition, when the robotic arm coordinate system is used, the first conversion parameters can still be used when the surgical instrument at the end of the robotic arm is replaced.

[0086] It should be noted that if the surgical instrument is located in the imaging field of the C-arm imaging system when photographing at the first photographing position and the second photographing position, calibration can be directly performed using the two medical images.

[0087] For details, please refer to Figure 7 , Figure 7 A schematic diagram of the implementation flow of another calibration method provided in an embodiment of the present application is shown, and the method can be applied to medical equipment.

[0088] Specifically, the above calibration method may include the following steps S701 to S706.

[0089] Step S701, when the surgical instrument is located in the imaging field of the C-arm imaging system, a fifth medical image and a sixth medical image are acquired by respectively photographing the target object by the C-arm imaging system in a first shooting posture and a second shooting posture.

[0090] Step S702: determining a first coordinate transformation matrix between the first shooting posture and the second shooting posture of the C-arm imaging system according to the fifth medical image and the sixth medical image.

[0091] Step S703: Using the C-arm imaging system coordinate system corresponding to the fifth medical image as the world coordinate system, and acquiring a first projection matrix of the fifth medical image.

[0092] Step S704: Obtain a second projection matrix of the sixth medical image according to the first projection matrix and the first coordinate conversion matrix.

[0093] Specifically, the fifth medical image and the sixth medical image are regarded as the first medical image and the second medical image respectively. The implementation method of steps S701 to S704 can refer to the description of steps S101 to S104. The only difference is that the surgical instruments in the fifth medical image and the sixth medical image are located in the imaging field of view of the C-arm imaging system, which is not elaborated in this application.

[0094] Step S705 , determining first coordinates of instrument feature points of the surgical instruments in the fifth medical image and the sixth medical image in the world coordinate system according to the first projection matrix and the second projection matrix.

[0095] Step S706, determining a first conversion parameter between the robotic arm coordinate system and the C-arm imaging system coordinate system according to the first coordinate and the second coordinate of the instrument feature point in the robotic arm coordinate system.

[0096] Specifically, the fifth medical image and the sixth medical image are regarded as the third medical image and the fourth medical image respectively. The implementation method of step S705 to step S706 can refer to the description of step S106 to step S107, which is not elaborated in this application.

[0097] Understandably, Figure 7 The calibration method shown is compared to Figure 1 The calibration method shown, since the surgical instrument is located in the imaging field of view of the C-arm imaging system when shooting the fifth medical image and the sixth medical image, there is no need to reshoot two new medical images (i.e., the third medical image and the fourth medical image). By shooting two medical images, the calibration can be completed in real time without the calibration target. This method does not require a large amount of calibration target data under different shooting postures before use, nor does it require conversion parameters obtained by interpolation. More accurate conversion parameters can be obtained based on the on-site installation of the C-arm imaging system and its own structural dimensions. In addition, by using the robotic arm coordinate system, the first conversion parameter can still be used when the surgical instrument at the end of the robotic arm is replaced.

[0098] In some embodiments of the present application, determining a first coordinate transformation matrix between a first shooting posture and a second shooting posture of a C-arm imaging system based on a fifth medical image and a sixth medical image may include: determining matching feature point pairs in the fifth medical image and the sixth medical image; and determining, based on the pixel positions of the feature point pairs, a first coordinate transformation matrix of the C-arm imaging system coordinate system of the sixth medical image relative to the C-arm imaging system coordinate system of the fifth medical image.

[0099] Specifically, according to the pixel position of the first feature point on the fifth medical image and the pixel position of the second feature point on the sixth medical image in the feature point pair, the eight-point method can be used to find the essential matrix E. The essential matrix E is used to describe the relationship between the coordinates of a point in space in the camera coordinate system under different shooting postures. According to the essential matrix E, the first coordinate transformation matrix of the C-arm imaging system coordinate system of the sixth medical image relative to the C-arm imaging system coordinate system of the fifth medical image can be found by singular value decomposition, and the first coordinate transformation matrix can be expressed as a rotation matrix R and a translation matrix t.

[0100] In some embodiments of the present application, determining the first coordinates of the instrument feature points of the surgical instrument in the fifth medical image and the sixth medical image in the world coordinate system based on the first projection matrix and the second projection matrix can include: obtaining at least three third feature points of the surgical instrument in the fifth medical image, obtaining at least three fourth feature points of the surgical instrument in the sixth medical image corresponding to the third feature points; obtaining the first coordinates of the instrument feature points of the surgical instrument in the world coordinate system based on the first projection matrix, the second projection matrix, the third feature points and the fourth feature points.

[0101] Specifically, when the first projection matrix of the fifth medical image and the second projection matrix of the sixth medical image are both known, the pixel position of the third feature point in the fifth medical image and the pixel position of the fourth feature point in the sixth medical image can be used to jointly solve the first coordinate of the instrument feature point of the surgical instrument in the world coordinate system.

[0102] Please refer to Figure 8 , Figure 8 A schematic diagram of the implementation flow of a surgical navigation method provided in an embodiment of the present application is shown. The method can be applied to a medical device, which can be a computer, a server or other intelligent device, and the present application does not impose any limitation on this.

[0103] Specifically, the above surgical navigation method may include the following steps S801 to S803.

[0104] Step S801, obtaining a first conversion parameter.

[0105] The method for obtaining the first conversion parameter can refer to Figures 1 to 6 For instructions on the calibration method shown, or refer to Figure 7 The description of the calibration method shown is not repeated in this application.

[0106] Step S802: displaying the target medical image captured by the C-arm imaging system.

[0107] The target medical image may be any one or more medical images captured by a C-arm imaging system.

[0108] Step S803: displaying the surgical instrument in the target medical image according to the real-time coordinates of the surgical instrument in the robot arm coordinate system and the first conversion parameter.

[0109] In the implementation of the present application, the surgical instrument can be adjusted to any position by the robotic arm during the operation, and the real-time coordinates of the surgical instrument in the robotic arm coordinate system can be obtained according to the control instructions of the robotic arm. According to the first conversion parameter, the real-time coordinates of the surgical instrument in the robotic arm coordinate system can be converted into the coordinates of the C-arm imaging system coordinate system, so as to determine the pixel position of the surgical instrument on the target medical image, so that the surgical instrument can be displayed in the target medical image.

[0110] This application does not limit the way surgical instruments are presented in the target medical image. For example, surgical instruments can be displayed in the target medical image in the form of dots, triangles, or patterns of surgical instruments.

[0111] In the implementation manner of the present application, on the one hand, the first conversion parameter is more accurate than the parameter obtained by conventional calibration. Surgical navigation based on the first conversion parameter can ensure the reliability of surgical navigation and help optimize the doctor's surgical decisions. On the other hand, the calibration process and the surgical navigation process can be integrated. When the patient is on the operating bed, the calibration method provided in the present application can be executed on the patient as the target object, and surgical navigation can be performed based on the first conversion parameter obtained by calibration, so as to achieve the effect of real-time calibration and rapid surgical navigation.

[0112] In some embodiments of the present application, the target medical image may be the first medical image and / or the second medical image. Since there are no surgical instruments in the first medical image and the second medical image, the surgical instruments can be displayed on the first medical image and the second medical image without removing the original surgical instruments, which helps to improve processing efficiency. In addition, by displaying medical images obtained from different shooting postures, it is convenient for doctors to obtain depth information of surgical instruments.

[0113] In some other embodiments of the present application, before displaying the target medical image captured by the C-arm imaging system, it may also include: controlling the C-arm imaging system to re-shoot the medical image as the target medical image. Specifically, if the doctor is not satisfied with the clarity or shooting posture of the first medical image and the second medical image, a new medical image may be re-shot for display. The new medical image may be one or more.

[0114] At this time, displaying the surgical instrument in the target medical image based on the real-time coordinates of the surgical instrument in the robotic arm coordinate system and the first conversion parameter may include: determining the second coordinate conversion matrix between the target medical image and the first medical image, and displaying the surgical instrument in the target medical image based on the real-time coordinates, the first conversion parameter and the second coordinate conversion matrix.

[0115] Specifically, the process of obtaining the second coordinate transformation matrix between the target medical image and the first medical image can refer to the aforementioned process of obtaining the first coordinate transformation matrix, that is, when the shooting posture of the target medical image is known, the essential matrix is ​​obtained by the eight-point method, and then the second coordinate transformation matrix is ​​obtained by singular value decomposition based on the essential matrix. In other implementations, if the image content of the target medical image is similar to that of the first medical image, the second coordinate transformation matrix can also be obtained by two-dimensional image registration.

[0116] The second coordinate conversion matrix can be used to perform coordinate conversion between the first medical image and the target medical image. The pixel position of the surgical instrument in the first medical image can be determined based on the real-time coordinates of the surgical instrument in the robot arm coordinate system and the first conversion parameter. The second coordinate conversion matrix can convert the pixel position of the surgical instrument in the first medical image into the pixel position of the surgical instrument on the target medical image, and then display the surgical instrument in the target medical image.

[0117] It should be noted that, for the sake of simplicity of description, the aforementioned method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the described order of actions, because according to the present application, certain steps can be performed in other orders.

[0118] like Fig. 9 Shown is a schematic structural diagram of a calibration device 900 provided in an embodiment of the present application, wherein the calibration device 900 is configured on a medical device.

[0119] Specifically, the calibration device 900 may include:

[0120] The first image acquisition unit 901 is used to acquire a first medical image and a second medical image obtained by respectively photographing a target object by a C-arm imaging system in a first shooting posture and a second shooting posture;

[0121] A coordinate conversion matrix determining unit 902 is used to determine a first coordinate conversion matrix between the first shooting posture and the second shooting posture of the C-arm imaging system according to the first medical image and the second medical image;

[0122] A first projection matrix acquisition unit 903 is used to use the C-arm imaging system coordinate system corresponding to the first medical image as the world coordinate system to acquire a first projection matrix of the first medical image;

[0123] A second projection matrix acquisition unit 904, configured to obtain a second projection matrix of the second medical image according to the first projection matrix and the first coordinate transformation matrix;

[0124] A second image acquisition unit 905 is used to acquire a third medical image captured by the C-arm imaging system in a third shooting posture and a fourth medical image captured in a fourth shooting posture when the surgical instrument is located in the imaging field of view of the C-arm imaging system;

[0125] A world coordinate acquisition unit 906, configured to determine first coordinates of the instrument feature points of the surgical instrument in the third medical image and the fourth medical image in the world coordinate system according to the first projection matrix and the second projection matrix;

[0126] The calibration unit 907 is used to determine a first conversion parameter between the robotic arm coordinate system and the C-arm imaging system coordinate system according to the first coordinate and the second coordinate of the instrument feature point in the robotic arm coordinate system of the robotic arm.

[0127] In some embodiments of the present application, the coordinate transformation matrix determination unit 902 is specifically used to: determine matching feature point pairs in the first medical image and the second medical image; and determine, based on the pixel positions of the feature point pairs, a first coordinate transformation matrix of the C-arm imaging system coordinate system of the second medical image relative to the C-arm imaging system coordinate system of the first medical image.

[0128] In some embodiments of the present application, the third shooting posture is the same as the first shooting posture, and the fourth shooting posture is the same as the second shooting posture. The world coordinate acquisition unit 906 is specifically used to: acquire at least three third feature points of the surgical instrument in the third medical image, acquire at least three fourth feature points of the surgical instrument in the fourth medical image corresponding to the third feature points; and obtain the first coordinates of the instrument feature points of the surgical instrument in the world coordinate system according to the first projection matrix, the second projection matrix, the third feature points and the fourth feature points.

[0129] In some embodiments of the present application, the world coordinate acquisition unit 906 is specifically used to: acquire at least three third feature points of the surgical instrument in the third medical image, and acquire at least three fourth feature points of the surgical instrument in the fourth medical image corresponding to the third feature points; align the third medical image with the first medical image to obtain a first registration matrix; align the fourth medical image with the second medical image to obtain a second registration matrix; and obtain the first coordinates of the instrument feature point of the surgical instrument in the world coordinate system according to the first registration matrix, the second registration matrix, the first projection matrix, the second projection matrix, the third feature point and the fourth feature point.

[0130] In some embodiments of the present application, the second shooting posture is obtained by controlling the C-arm imaging system to translate along a preset direction and a preset distance from the first shooting posture.

[0131] In some embodiments of the present application, the preset direction is a direction perpendicular to the sagittal plane of the target object.

[0132] It should be noted that for the convenience and simplicity of description, the specific working process of the above-mentioned calibration device 900 can refer to Figures 1 to 6 The corresponding process of the method will not be repeated here.

[0133] like Fig.10 Shown is a schematic structural diagram of a calibration device 1000 provided in an embodiment of the present application, wherein the calibration device 1000 is configured on a medical device.

[0134] Specifically, the calibration device 1000 may include:

[0135] The third image acquisition unit 1001 is used to acquire a fifth medical image and a sixth medical image obtained by respectively photographing the target object by the C-arm imaging system in the first shooting posture and the second shooting posture when the surgical instrument is located in the imaging field of view of the C-arm imaging system;

[0136] A coordinate conversion matrix determining unit 1002, configured to determine a first coordinate conversion matrix between the first shooting posture and the second shooting posture of the C-arm imaging system according to the fifth medical image and the sixth medical image;

[0137] A first projection matrix acquisition unit 1003 is used to use the C-arm imaging system coordinate system corresponding to the fifth medical image as the world coordinate system to acquire a first projection matrix of the fifth medical image;

[0138] A second projection matrix acquiring unit 1004, configured to obtain a second projection matrix of the sixth medical image according to the first projection matrix and the first coordinate transformation matrix;

[0139] A world coordinate acquisition unit 1005, configured to determine first coordinates of the instrument feature points of the surgical instrument in the fifth medical image and the sixth medical image in the world coordinate system according to the first projection matrix and the second projection matrix;

[0140] The calibration unit 1006 is used to determine a first conversion parameter between the robotic arm coordinate system and the C-arm imaging system coordinate system according to the first coordinate and the second coordinate of the instrument feature point in the robotic arm coordinate system of the robotic arm.

[0141] In some embodiments of the present application, the coordinate transformation matrix determination unit 1002 can be specifically used to: determine matching feature point pairs in the fifth medical image and the sixth medical image; and determine, based on the pixel positions of the feature point pairs, a first coordinate transformation matrix of the C-arm imaging system coordinate system of the sixth medical image relative to the C-arm imaging system coordinate system of the fifth medical image.

[0142] In some embodiments of the present application, the world coordinate acquisition unit 1005 can be specifically used to: obtain at least three third feature points of the surgical instrument in the fifth medical image, and obtain at least three fourth feature points of the surgical instrument in the sixth medical image corresponding to the third feature points; and obtain the first coordinates of the instrument feature points of the surgical instrument in the world coordinate system according to the first projection matrix, the second projection matrix, the third feature points and the fourth feature points.

[0143] It should be noted that for the convenience and simplicity of description, the specific working process of the above-mentioned calibration device 900 can refer to Figure 7 The corresponding process of the method will not be repeated here.

[0144] like Fig.11 FIG. 1 is a schematic diagram of the structure of a surgical navigation device 1100 provided in an embodiment of the present application. The surgical navigation device 1100 is configured on a medical device.

[0145] Specifically, the surgical navigation device 1100 may include:

[0146] The calibration unit 1101 is used to obtain a first conversion parameter;

[0147] An image display unit 1102 displays a target medical image captured by a C-arm imaging system;

[0148] The instrument display unit 1103 displays the surgical instrument in the target medical image according to the real-time coordinates of the surgical instrument in the robot arm coordinate system and the first conversion parameters.

[0149] In some embodiments of the present application, the target medical image is the first medical image and / or the second medical image.

[0150] In some embodiments of the present application, the surgical navigation device 1100 also includes an image acquisition unit for controlling the C-arm imaging system to re-shoot the medical image as the target medical image; the instrument display unit 1103 is specifically used to: determine a second coordinate transformation matrix between the target medical image and the first medical image, and display the surgical instrument in the target medical image according to the real-time coordinates, the first transformation parameters and the second coordinate transformation matrix.

[0151] It should be noted that, for the convenience and simplicity of description, the specific working process of the surgical navigation device 1100 can be referred to Figure 8 The corresponding process of the method will not be repeated here.

[0152] like Fig.12 , which is a schematic diagram of a medical device 12 provided in an embodiment of the present application. Specifically, the medical device 12 may include: a processor 120, a memory 121, and a computer program 122 stored in the memory 121 and executable on the processor 120, such as a calibration program.

[0153] When the processor 120 executes the computer program 122, the steps in the above-mentioned calibration method embodiments are implemented, for example: Figure 1 Alternatively, when the processor 120 executes the computer program 122, the steps in the above-mentioned calibration method embodiments are implemented, for example, Figure 7 Alternatively, when the processor 120 executes the computer program 122, the steps in the above-mentioned various surgical navigation method embodiments are implemented, for example Figure 8 Steps S801 to S803 are shown.

[0154] Alternatively, when the processor 120 executes the computer program 122, the functions of the modules / units in the above-mentioned device embodiments are realized, for example Fig. 9 The functions of the first image acquisition unit 901, the coordinate transformation matrix determination unit 902, the first projection matrix acquisition unit 903, the second projection matrix acquisition unit 904, the second image acquisition unit 905, the world coordinate acquisition unit 906 and the calibration unit 907 are shown. Alternatively, when the processor 120 executes the computer program 122, the functions of each module / unit in the above-mentioned device embodiments are realized, for example Fig.10 The functions of the third image acquisition unit 1001, the coordinate transformation matrix determination unit 1002, the first projection matrix acquisition unit 1003, the second projection matrix acquisition unit 1004, the world coordinate acquisition unit 1005 and the calibration unit 1006 are shown. Alternatively, when the processor 120 executes the computer program 122, the functions of each module / unit in the above-mentioned device embodiments are realized, for example Fig.11 The functions of the calibration unit 1101, the image display unit 1102 and the instrument display unit 1103 are shown.

[0155] The computer program may be divided into one or more modules / units, which are stored in the memory 121 and executed by the processor 120 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the medical device 12.

[0156] The medical device 12 may include, but is not limited to, a processor 120 and a memory 121. Those skilled in the art will appreciate that Fig.12 It is only an example of the medical device 12 and does not constitute a limitation of the medical device 12. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the medical device 12 may also include input and output devices, network access devices, buses, etc.

[0157] The processor 120 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0158] The memory 121 may be an internal storage unit of the medical device 12, such as a hard disk or memory of the medical device 12. The memory 121 may also be an external storage device of the medical device 12, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the medical device 12. Further, the memory 121 may also include both an internal storage unit and an external storage device of the medical device 12. The memory 121 is used to store the computer program and other programs and data required by the medical device 12. The memory 121 may also be used to temporarily store data that has been output or is to be output.

[0159] It should be noted that, for the convenience and brevity of description, the structure of the above-mentioned medical device 12 can also refer to the specific description of the structure in the method embodiment, and will not be repeated here.

[0160] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0161] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0162] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0163] In the embodiments provided in the present application, it should be understood that the disclosed devices / equipment and methods can be implemented in other ways. For example, the device / equipment embodiments described above are merely schematic, for example, the division of the modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0164] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0165] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0166] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0167] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A calibration method, characterized in that: include: Acquire a first medical image and a second medical image obtained by photographing the target object by the C-arm imaging system in a first photographing posture and a second photographing posture respectively; Determine a first coordinate transformation matrix between the first shooting posture and the second shooting posture of the C-arm imaging system according to the first medical image and the second medical image; Using the C-arm imaging system coordinate system corresponding to the first medical image as the world coordinate system, obtaining a first projection matrix of the first medical image; Obtaining a second projection matrix of the second medical image according to the first projection matrix and the first coordinate transformation matrix; When the surgical instrument is located in the imaging field of the C-arm imaging system, acquiring a third medical image captured by the C-arm imaging system in a third shooting posture and a fourth medical image captured in a fourth shooting posture; Determine, according to the first projection matrix and the second projection matrix, first coordinates of instrument feature points of the surgical instrument in the third medical image and the fourth medical image in the world coordinate system; A first conversion parameter between the robotic arm coordinate system and the C-arm imaging system coordinate system is determined according to the first coordinate and the second coordinate of the instrument feature point in the robotic arm coordinate system.

2. The calibration method according to claim 1, characterized in that: The step of determining a first coordinate transformation matrix between the first shooting posture and the second shooting posture of the C-arm imaging system according to the first medical image and the second medical image includes: Determining matching feature point pairs in the first medical image and the second medical image; According to the pixel positions of the feature point pairs, a first coordinate transformation matrix of the C-arm imaging system coordinate system of the second medical image relative to the C-arm imaging system coordinate system of the first medical image is determined.

3. The calibration method according to claim 1, characterized in that: The third shooting posture is the same as the first shooting posture, and the fourth shooting posture is the same as the second shooting posture; Determining first coordinates of instrument feature points of the surgical instrument in the third medical image and the fourth medical image in the world coordinate system according to the first projection matrix and the second projection matrix includes: Acquire at least three third feature points of the surgical instrument in the third medical image, and acquire at least three fourth feature points of the surgical instrument in the fourth medical image corresponding to the third feature points; According to the first projection matrix, the second projection matrix, the third feature point and the fourth feature point, a first coordinate of the instrument feature point of the surgical instrument in the world coordinate system is obtained.

4. The calibration method according to claim 1, characterized in that: Determining first coordinates of instrument feature points of the surgical instrument in the third medical image and the fourth medical image in the world coordinate system according to the first projection matrix and the second projection matrix includes: Acquire at least three third feature points of the surgical instrument in the third medical image, and acquire at least three fourth feature points of the surgical instrument in the fourth medical image corresponding to the third feature points; Registering the third medical image with the first medical image to obtain a first registration matrix; Registering the fourth medical image with the second medical image to obtain a second registration matrix; According to the first registration matrix, the second registration matrix, the first projection matrix, the second projection matrix, the third feature point and the fourth feature point, the first coordinates of the instrument feature point of the surgical instrument in the world coordinate system are obtained.

5. The calibration method according to any one of claims 1 to 4, characterized in that: The second shooting posture is obtained by controlling the C-arm imaging system to translate along a preset direction and a preset distance from the first shooting posture.

6. The calibration method according to claim 5, characterized in that: The preset direction is a direction perpendicular to the sagittal plane of the target object.

7. A calibration method, characterized in that: include: When the surgical instrument is located in the imaging field of view of the C-arm imaging system, a fifth medical image and a sixth medical image are acquired by respectively photographing the target object by the C-arm imaging system in the first photographing posture and the second photographing posture; Determine a first coordinate transformation matrix between the first shooting posture and the second shooting posture of the C-arm imaging system according to the fifth medical image and the sixth medical image; Using the C-arm imaging system coordinate system corresponding to the fifth medical image as the world coordinate system, obtaining a first projection matrix of the fifth medical image; Obtaining a second projection matrix of the sixth medical image according to the first projection matrix and the first coordinate transformation matrix; Determine, according to the first projection matrix and the second projection matrix, first coordinates of instrument feature points of the surgical instrument in the fifth medical image and the sixth medical image in the world coordinate system; A first conversion parameter between the robotic arm coordinate system and the C-arm imaging system coordinate system is determined according to the first coordinate and the second coordinate of the instrument feature point in the robotic arm coordinate system.

8. The calibration method according to claim 7, characterized in that: The determining a first coordinate conversion matrix between the first shooting posture and the second shooting posture of the C-arm imaging system according to the fifth medical image and the sixth medical image includes: Determining matching feature point pairs in the fifth medical image and the sixth medical image; According to the pixel positions of the feature point pairs, a first coordinate transformation matrix of the C-arm imaging system coordinate system of the sixth medical image relative to the C-arm imaging system coordinate system of the fifth medical image is determined.

9. The calibration method according to claim 7, characterized in that: Determining first coordinates of instrument feature points of the surgical instrument in the fifth medical image and the sixth medical image in the world coordinate system according to the first projection matrix and the second projection matrix includes: Acquire at least three third feature points of the surgical instrument in the fifth medical image, and acquire at least three fourth feature points of the surgical instrument in the sixth medical image corresponding to the third feature points; According to the first projection matrix, the second projection matrix, the third feature point and the fourth feature point, a first coordinate of the instrument feature point of the surgical instrument in the world coordinate system is obtained.

10. A surgical navigation method, characterized in that: include: Obtaining a first conversion parameter, where the first conversion parameter is obtained according to the calibration method according to any one of claims 1 to 6 or the calibration method according to any one of claims 7 to 9; Display the target medical image captured by the C-arm imaging system; The surgical instrument is displayed in the target medical image according to the real-time coordinates of the surgical instrument in the robot arm coordinate system and the first conversion parameters.

11. The surgical navigation method according to claim 10, characterized in that: The target medical image is the first medical image and / or the second medical image.

12. The surgical navigation method according to claim 10, wherein: Before displaying the target medical image captured by the C-arm imaging system, the surgical navigation method further includes: controlling the C-arm imaging system to re-capture a medical image as the target medical image; The method of displaying the surgical instrument in the target medical image based on the real-time coordinates of the surgical instrument in the robot arm coordinate system and the first conversion parameters includes: determining a second coordinate conversion matrix between the target medical image and the first medical image, and displaying the surgical instrument in the target medical image based on the real-time coordinates, the first conversion parameters and the second coordinate conversion matrix.

13. A medical device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the calibration method as described in any one of claims 1 to 6 are implemented; or, when the processor executes the computer program, the steps of the calibration method as described in any one of claims 7 to 9 are implemented; or, when the processor executes the computer program, the steps of the surgical navigation method as described in any one of claims 10 to 12 are implemented.

14. A computer program product, characterized in that It comprises a computer program, which, when being run, enables the calibration method as described in any one of claims 1 to 6 to be executed, or, when being run, enables the calibration method as described in any one of claims 7 to 9 to be executed, or, when being run, enables the surgical navigation method as described in any one of claims 10 to 12 to be executed.

Images