CT image registration method and system for real-time projection

Abstract

The invention provides a CT image registration method and system for real-time projection, and belongs to the field of image processing, and the method comprises the steps: adjusting the position of a CT image of a target under a projection coordinate system according to the real-time position change amount of the target and the conversion relation between the projection coordinate system and a CT coordinate system, so as to enable the projection image of the CT image to move along with the target, cT image real-time projection is realized; the method for obtaining the conversion relation between the projection coordinate system and the CT coordinate system comprises the steps that the relative positions of the projection system, the projection screen and the shooting system are set, and the conversion relation between the projection coordinate system and an image coordinate system is obtained; and converting the reference object image under the image coordinate system into a reference object image under the projection coordinate system, and matching the reference object image with the CT image of the reference object to obtain a conversion relation between the projection coordinate system and the CT coordinate system. The position of each structure in the target can be determined in real time through the CT image.

Description

Technical Field

[0001] The present application belongs to the field of image processing, and more specifically, to a CT image registration method and system for real-time projection. Background Art

[0002] Surgical navigation systems assist different surgeries to improve surgical accuracy and reduce the risk of radiation exposure to patients. Existing minimally invasive abdominal surgeries are generally performed in vivo through laparoscopes under the guidance of preliminary CT examination results. Limited by the field of view and small operating space of the laparoscope, doctors only rely on the preliminary CT images to determine the location of the lesion, which often requires long-term exploration or real-time imaging assistance from online CT equipment, causing more pain and unnecessary radiation doses to patients.

[0003] Existing surgical navigation assistance systems mostly track by implanting characteristic materials and using head-mounted devices for virtual imaging, and are mostly used in orthopedic and neurosurgery surgeries involving rigid parts. The gap between lesion localization in abdominal surgery and bone tracking technology is large, and the development cost of head-mounted devices is high, so existing technologies cannot be directly transplanted into abdominal surgery. Summary of the invention

[0004] In view of the defects of the prior art, the purpose of this application is to provide a CT image registration method and system for real-time projection, aiming to solve the problem that due to the limitation of the field of view of the laparoscope and the small operating space, long-term exploration or real-time imaging assistance of online CT equipment is often required to determine the positions of various structures inside the target through CT images.

[0005] To achieve the above objectives, in a first aspect, the present application provides a CT image registration method for real-time projection, comprising the following steps:

[0006] According to the real-time position change of the target and the conversion relationship between the projection coordinate system and the CT coordinate system, the CT image position of the target is adjusted in the projection coordinate system so that the projection image of the CT image follows the movement of the target, thereby realizing the real-time projection of the CT image;

[0007] The method for obtaining the conversion relationship between the projection coordinate system and the CT coordinate system specifically includes the following steps:

[0008] Step S1: setting the relative positions of the projection system, the projection screen and the camera system, and obtaining the conversion relationship between the projection coordinate system and the image coordinate system; wherein the projection coordinate system is the two-dimensional image coordinate system projected by the projection system; and the image coordinate system is the two-dimensional image coordinate system captured by the camera system;

[0009] Step S2: Based on the conversion relationship between the projection coordinate system and the image coordinate system, the reference object image in the image coordinate system is converted into the reference object image in the projection coordinate system, and matched with the CT image of the reference object to obtain the conversion relationship between the projection coordinate system and the CT coordinate system.

[0010] Further preferably, before step S1, the method further includes the following steps:

[0011] A shooting system is used to obtain multiple sets of images of checkerboard calibration plates, corner points in the images are extracted, and the intrinsic parameter matrix, extrinsic parameter matrix and distortion parameters of the shooting system are calculated;

[0012] The intrinsic parameter matrix, the extrinsic parameter matrix and the distortion parameters are applied to the image captured by the shooting system to correct the lens distortion of the shooting system.

[0013] Further preferably, step S1 specifically comprises:

[0014] Setting the relative positions of the projection system, the projection screen and the camera system so that all the contents projected by the projection system can be captured by the camera system;

[0015] A checkerboard image is projected by a projection system or a printed checkerboard image is placed on the projection screen, and a photograph is taken by a camera system to obtain a standard image;

[0016] The standard image is projected again by the projection system, and photographed again by the photographing system to obtain a projection image of the standard image as the image to be matched;

[0017] The standard image is registered with the image to be registered, and the conversion relationship between the projection coordinate system and the image coordinate system is obtained.

[0018] Further preferably, the position change of the target is acquired by detecting the absolute movement of the target objective marker.

[0019] In a second aspect, the present application provides a CT image registration system for real-time projection, comprising:

[0020] A CT image projection control module is used to adjust the CT image position of the target in the projection coordinate system according to the real-time position change of the target and the conversion relationship between the projection coordinate system and the CT coordinate system, so that the projection image of the CT image moves with the target, thereby realizing real-time projection of the CT image;

[0021] A first coordinate conversion module is used to set the relative positions of the projection system, the projection screen and the shooting system, and obtain the conversion relationship between the projection coordinate system and the image coordinate system; wherein the projection coordinate system is the two-dimensional image coordinate system projected by the projection system; and the image coordinate system is the two-dimensional image coordinate system captured by the shooting system;

[0022] The second coordinate conversion module is used to convert the reference object image in the image coordinate system into the reference object image in the projection coordinate system based on the conversion relationship between the projection coordinate system and the image coordinate system, and match it with the CT image of the reference object to obtain the conversion relationship between the projection coordinate system and the CT coordinate system.

[0023] Further preferably, the CT image registration system also includes a shooting lens correction module, which is used to use the shooting system to obtain multiple groups of images of the checkerboard calibration plate, extract the corner points in the image, and calculate the intrinsic parameter matrix, extrinsic parameter matrix and distortion parameters of the shooting system; apply the intrinsic parameter matrix, extrinsic parameter matrix and distortion parameters to the image taken by the shooting system to correct the lens distortion of the shooting system.

[0024] Further preferably, the first coordinate conversion module includes a data processing unit, a shooting system and a projection system;

[0025] The projection system is used to project a checkerboard image, or a printed checkerboard image is placed on its projection screen;

[0026] The shooting system is used to shoot a checkerboard image to obtain a standard image;

[0027] The projection system is used to project a standard image;

[0028] The shooting system is used to shoot the projected standard image again to obtain a projected image of the standard image as the image to be matched;

[0029] The data processing unit is used to register the standard image with the image to be registered, and obtain the conversion relationship between the projection coordinate system and the image coordinate system.

[0030] Further preferably, the CT image projection control module is used to obtain the position change of the target by detecting the absolute movement of the target objective marker.

[0031] In general, the above technical solutions conceived by this application have the following beneficial effects compared with the prior art:

[0032] The present application provides a CT image registration method for real-time projection, which can adjust the CT image position of the target in the projection coordinate system according to the real-time position change of the target and the conversion relationship between the projection coordinate system and the CT coordinate system, so that the projection image of the CT image can follow the movement of the target, and can focus on abdominal surgery. The existing technology mainly focuses on orthopedics and neurosurgery. Since the existing technology requires the surgical target to be rigid, the scope of application is relatively limited. The CT image registration method provided in the present application can expand the scope of application of the surgical navigation system.

[0033] The CT image registration method for real-time projection provided in the present application adopts a contour-based registration method (adjusting the CT image position of the target in the projection coordinate system according to the real-time position change of the target and the conversion relationship between the projection coordinate system and the CT coordinate system). Compared with the prior art which adopts a feature point-based registration method and needs to implant markers into specific parts of the patient's body, the present application does not rely on special substances and can greatly reduce secondary damage to patients.

[0034] The present application provides a CT image registration method for real-time projection, which uses a projector to project and register CT images. Most of the existing technologies use virtual reality surgical microscopes and head-mounted displays, and the purchase and maintenance costs of the equipment are high. However, the present application only requires a simple projector to complete the process, which reduces the implementation conditions and usage costs of the technology.

[0035] This application proposes a CT image registration system for real-time projection, including the entire registration process between the camera, projector and human body (target). Currently, most technologies only propose a single technical link and lack a systematic idea. This application realizes the complete process from equipment debugging to tracking correction, and forms a user-friendly operation interface, providing a convenient and feasible new idea for intraoperative lesion localization. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is a flow chart of a system calibration method provided in an embodiment of the present application;

[0037] Figure 2 It is a schematic diagram of the system construction used for acquiring two sets of calibration images provided in the embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0039] The term "and / or" in this article is a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The symbol " / " in this article indicates that the associated objects are in an or relationship, for example, A / B means A or B.

[0040] The terms “first”, “second” and the like in the specification and claims herein are used to distinguish different objects rather than to describe a specific order of the objects.

[0041] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0042] In the description of the embodiments of the present application, unless otherwise specified, “plurality” means two or more than two.

[0043] The technical solutions provided in the embodiments of the present application are introduced below.

[0044] Example 1

[0045] The CT image registration system for real-time projection provided by this application focuses on abdominal surgery and has the characteristics of low cost and strong specialization. The system includes: camera, projector and image processing module. The above devices are all in a relatively mature market, and the cost and difficulty of acquisition are much lower than other surgical auxiliary systems.

[0046] This application intends to use technologies such as image recognition, spatial registration, and image projection matching to build a complete structured light system and adopt a contour-based registration method to break the limitation that characteristic substances in the abdominal cavity are difficult to locate and track, simplify preoperative preparation and postoperative operations, assist doctors in grasping the location of lesions more accurately during surgery, and project the patient's CT image onto the patient's body surface in real time through a projector, providing accurate and intuitive location information for abdominal surgery.

[0047] The platform construction requires: a camera, a projector and a computer; the camera is used to obtain the actual position image of the object in real time; the projector is used to project the CT auxiliary image; the projector is connected to the computer; the operating table is used to provide the operating space; the computer is used for image analysis and matching; the object here refers to the personnel who check the CT in this embodiment, and the image processing module in this embodiment adopts a computer, but is not limited to a computer;

[0048] The following is the specific operation process of the computer:

[0049] The camera and projector form a structured light system; the structured light system needs to adjust the parameters of the camera and projector, such as field of view, magnification and focal length, according to the size of the object and the measurement distance; therefore, it should be calibrated before use in surgery. Calibration with sufficient accuracy is the prerequisite for controlling the image projection position. The simplification of the system calibration process and the accuracy of the system are equally important; it is necessary to complete the conversion between the three coordinate systems designed in the system (image coordinate system, projector coordinate system and CT coordinate system; among them, the image coordinate system is the two-dimensional image coordinate system acquired by the camera; the projector coordinate system is the two-dimensional image coordinate system projected by the projector; the CT coordinate system is the two-dimensional coordinate system of the CT image). The calibration of the system requires the design of a joint calibration method for the projector and the camera. Through the study and research of the calibration principle and the camera imaging principle, the dual devices of the whole system are calibrated based on the principle of Zhang Zhengyou's chessboard calibration method. The specific method of system calibration is as follows: Figure 1 As shown, the specific steps include:

[0050] Step 1: Use a camera to obtain multiple sets of images of the checkerboard calibration plate, extract the corner points in the image, and calculate the camera's intrinsic parameter matrix, extrinsic parameter matrix and distortion parameters. Apply the intrinsic parameter matrix, extrinsic parameter matrix and distortion parameters to the image taken by the camera to correct the camera's lens distortion problem to obtain a correct and complete image.

[0051] Step 2: If Figure 2 As shown, the relative positions of the projector, projection screen and camera are reasonably set so that all the contents projected by the projector can be acquired by the camera; a chessboard image is projected by the projector or a printed chessboard image is placed on the projection screen, and the image is photographed by the rectified camera, and the image photographed by the camera is called the standard image; the standard image is projected again by the projector, photographed by the camera, and the projection image of the standard image is obtained, and the image is called the image to be matched; the standard image is registered with the image to be matched, and the transformation matrix between the two images is obtained, that is, the transformation relationship between the projection coordinate system and the image coordinate system is obtained;

[0052] Step 3: Place the human model on the projection screen, shoot with a camera, process the human image acquired by the camera with the conversion matrix obtained in step 2, obtain the human image in the projector coordinate system, match the pre-acquired CT image with the image, obtain the conversion matrix between the two images, that is, obtain the conversion relationship between the projector coordinate system and the CT coordinate system, and project the registered CT image with a projector. At this time, the organ information on the CT image is presented at the correct position on the body surface of the human model, realizing the visualization of the organ information;

[0053] Step 4: Keep the phase position of the projector and camera unchanged and make fine adjustments to the position of the human body.

[0054] The present application obtains the position change of the human body by detecting the absolute movement of objective markers on the human body, processes the position of the CT image in the projection coordinate system according to the change of the human body position, and realizes the projection image following the movement of the human body, that is, realizing real-time projection.

[0055] Example 2

[0056] The present application provides a CT image registration system for real-time projection, comprising:

[0057] A CT image projection control module is used to adjust the CT image position of the target in the projection coordinate system according to the real-time position change of the target and the conversion relationship between the projection coordinate system and the CT coordinate system, so that the projection image of the CT image moves with the target, thereby realizing real-time projection of the CT image;

[0058] A first coordinate conversion module is used to set the relative positions of the projection system, the projection screen and the shooting system, and obtain the conversion relationship between the projection coordinate system and the image coordinate system; wherein the projection coordinate system is the two-dimensional image coordinate system projected by the projection system; and the image coordinate system is the two-dimensional image coordinate system captured by the shooting system;

[0059] The second coordinate conversion module is used to convert the reference object image in the image coordinate system into the reference object image in the projection coordinate system based on the conversion relationship between the projection coordinate system and the image coordinate system, and match it with the CT image of the reference object to obtain the conversion relationship between the projection coordinate system and the CT coordinate system.

[0060] Further preferably, the CT image registration system also includes a shooting lens correction module, which is used to use the shooting system to obtain multiple groups of images of the checkerboard calibration plate, extract the corner points in the image, and calculate the intrinsic parameter matrix, extrinsic parameter matrix and distortion parameters of the shooting system; apply the intrinsic parameter matrix, extrinsic parameter matrix and distortion parameters to the image taken by the shooting system to correct the lens distortion of the shooting system.

[0061] Further preferably, the first coordinate conversion module includes a data processing unit, a shooting system and a projection system;

[0062] The projection system is used to project a checkerboard image, or a printed checkerboard image is placed on its projection screen;

[0063] The shooting system is used to shoot a checkerboard image to obtain a standard image;

[0064] The projection system is used to project a standard image;

[0065] The shooting system is used to shoot the projected standard image again to obtain a projected image of the standard image as the image to be matched;

[0066] The data processing unit is used to register the standard image with the image to be registered, and obtain the conversion relationship between the projection coordinate system and the image coordinate system.

[0067] Further preferably, the CT image projection control module is used to obtain the position change of the target by detecting the absolute movement of the target objective marker.

[0068] Compared with the prior art, this application has the following advantages:

[0069] The present application provides a CT image registration method for real-time projection, which can focus on abdominal surgery. The existing technology mainly focuses on orthopedics and neurosurgery. Since the existing technology requires the surgical target to be rigid, the application scope is relatively limited. The CT image registration method provided by the present application can expand the application scope of the surgical navigation system.

[0070] The CT image registration method for real-time projection provided in the present application adopts a contour-based registration method (adjusting the CT image position of the target in the projection coordinate system according to the real-time position change of the target and the conversion relationship between the projection coordinate system and the CT coordinate system). Compared with the prior art which adopts a feature point-based registration method and needs to implant markers into specific parts of the patient's body, the present application does not rely on special substances and can greatly reduce secondary damage to patients.

[0071] The present application provides a CT image registration method for real-time projection, which uses a projector to project and register CT images. Most of the existing technologies use virtual reality surgical microscopes and head-mounted displays, and the purchase and maintenance costs of the equipment are high. However, the present application only requires a simple projector to complete the process, which reduces the implementation conditions and usage costs of the technology.

[0072] This application proposes a CT image registration system for real-time projection, including the entire registration process between the camera, projector and human body (target). Currently, most technologies only propose a single technical link and lack a systematic idea. This application realizes the complete process from equipment debugging to tracking correction, and forms a user-friendly operation interface, providing a convenient and feasible new idea for intraoperative lesion localization.

[0073] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A CT image registration method for real-time projection, characterized in that: The following steps are involved: According to the real-time position change of the target and the conversion relationship between the projection coordinate system and the CT coordinate system, the CT image position of the target is adjusted in the projection coordinate system so that the projection image of the CT image follows the movement of the target, thereby realizing the real-time projection of the CT image; The method for obtaining the conversion relationship between the projection coordinate system and the CT coordinate system specifically includes the following steps: Step S1: setting the relative positions of the projection system, the projection screen and the camera system, and obtaining the conversion relationship between the projection coordinate system and the image coordinate system; wherein the projection coordinate system is the two-dimensional image coordinate system projected by the projection system; and the image coordinate system is the two-dimensional image coordinate system captured by the camera system; Step S2: Based on the conversion relationship between the projection coordinate system and the image coordinate system, the reference object image in the image coordinate system is converted into the reference object image in the projection coordinate system, and matched with the CT image of the reference object to obtain the conversion relationship between the projection coordinate system and the CT coordinate system.

2. The CT image registration method according to claim 1, characterized in that: Before step S1, the method further includes the following steps: A shooting system is used to obtain multiple sets of images of checkerboard calibration plates, corner points in the images are extracted, and the intrinsic parameter matrix, extrinsic parameter matrix and distortion parameters of the shooting system are calculated; The intrinsic parameter matrix, the extrinsic parameter matrix and the distortion parameters are applied to the image captured by the shooting system to correct the lens distortion of the shooting system.

3. The CT image registration method according to claim 1 or 2, characterized in that: Step S1 is specifically as follows: Setting the relative positions of the projection system, the projection screen and the camera system so that all the contents projected by the projection system can be captured by the camera system; A checkerboard image is projected by a projection system or a printed checkerboard image is placed on the projection screen, and a photograph is taken by a camera system to obtain a standard image; The standard image is projected again by the projection system, and photographed again by the photographing system to obtain a projection image of the standard image as the image to be matched; The standard image is registered with the image to be registered, and the conversion relationship between the projection coordinate system and the image coordinate system is obtained.

4. The CT image registration method according to claim 1, characterized in that: The position change of the target is obtained by detecting the absolute movement of the target's objective marker.

5. A CT image registration system for real-time projection, characterized in that: include: A CT image projection control module is used to adjust the CT image position of the target in the projection coordinate system according to the real-time position change of the target and the conversion relationship between the projection coordinate system and the CT coordinate system, so that the projection image of the CT image moves with the target, thereby realizing real-time projection of the CT image; A first coordinate conversion module is used to set the relative positions of the projection system, the projection screen and the shooting system, and obtain the conversion relationship between the projection coordinate system and the image coordinate system; wherein the projection coordinate system is the two-dimensional image coordinate system projected by the projection system; and the image coordinate system is the two-dimensional image coordinate system captured by the shooting system; The second coordinate conversion module is used to convert the reference object image in the image coordinate system into the reference object image in the projection coordinate system based on the conversion relationship between the projection coordinate system and the image coordinate system, and match the CT image of the reference object to obtain the conversion relationship between the projection coordinate system and the CT coordinate system.

6. The CT image registration system according to claim 5, characterized in that: It also includes a shooting lens correction module, which is used to use a shooting system to obtain multiple groups of chessboard calibration plate images, extract corner points in the image, and calculate the intrinsic parameter matrix, extrinsic parameter matrix and distortion parameters of the shooting system; the intrinsic parameter matrix, extrinsic parameter matrix and distortion parameters are applied to the image taken by the shooting system to correct the lens distortion of the shooting system.

7. The CT image registration system according to claim 5, characterized in that: The first coordinate conversion module includes a data processing unit, a shooting system and a projection system; The projection system is used to project a checkerboard image, or a printed checkerboard image is placed on its projection screen; The shooting system is used to shoot a checkerboard image to obtain a standard image; The projection system is used to project a standard image; The shooting system is used to shoot the projected standard image again to obtain a projected image of the standard image as the image to be matched; The data processing unit is used to register the standard image with the image to be registered, and obtain the conversion relationship between the projection coordinate system and the image coordinate system.

8. The CT image registration system according to any one of claims 5 to 7, characterized in that: The CT image projection control module is used to obtain the position change of the target by detecting the absolute movement of the target objective marker.

Images