Imaging equipment positioning guiding method, system and equipment based on sparse reconstruction

Abstract

The invention discloses an imaging equipment positioning guiding method, system and equipment based on sparse reconstruction, and the method comprises the steps: employing imaging equipment, and automatically collecting 2D images of a designated region of a simulation patient from a plurality of sparse angles; selecting a target tissue to be modeled on the 2D image; performing 3D modeling on the selected target tissue by adopting an extremely sparse reconstruction algorithm to form a 3D modeling image of the target tissue; registering the 3D modeling image and the 2D image, and calculating a first conversion relation between a visual coordinate system and a motion coordinate system of the imaging equipment; adjusting the initial pose of the 3D modeling image, and calculating the target position of the imaging device corresponding to the adjusted pose based on the first conversion relation and the adjusted pose of the 3D modeling image; and controlling the imaging equipment to reach the target position so as to obtain an expected imaging view angle of the target tissue.

Description

Technical Field

[0001] This application relates to the field of medical imaging technology. Specifically, it relates to a method, system, and device for guiding the positioning of an imaging device based on sparse reconstruction. Background Art

[0002] In the fields of clinical orthopedics, neurosurgery, and interventional radiology, accurate three-dimensional imaging information is of great significance for surgical planning, instrument guidance, and intraoperative navigation.

[0003] Currently, the imaging angles of movable imaging devices (radiological instruments) are mostly manually adjusted by doctors based on experience. Although this manual adjustment method can meet the basic imaging requirements, there are many inconveniences in actual operation. For example, manually adjusting the angle requires doctors to frequently move the device during the operation, which is not only time-consuming and laborious but may also result in unsatisfactory imaging effects. In addition, the accuracy of manual adjustment is limited. Especially when multiple-angle projections are required, repeatedly searching for the projection target will significantly reduce work efficiency and increase the radiation dose.

[0004] On the other hand, X-ray imaging is difficult to accurately reflect the positional relationship of objects in three-dimensional space. When surgical instruments (such as bone nails) and bones overlap during interventional surgery, it is difficult to accurately judge the relative positional relationship between the nail and the bone and whether it penetrates. Therefore, intraoperative multi-angle fluoroscopy or combined with three-dimensional imaging for auxiliary judgment at the expected angle is often required. Summary of the Invention

[0005] To solve the above technical problems, this application discloses a method, system, and device for guiding the positioning of an imaging device based on sparse reconstruction. By using a very small number of projection images, combined with technologies such as sparse reconstruction and image registration, a three-dimensional space model of the target tissue is quickly obtained, and combined with the target position of the surgical plan, precise positioning of the patient's target tissue and guiding the positioning of the imaging device are realized.

[0006] Specifically, the technical solution of this application is as follows:

[0007] In a first aspect, this application discloses a method for guiding the positioning of an imaging device based on sparse reconstruction, including the following steps:

[0008] Use the imaging device to automatically collect 2D images of the simulated specified area of the patient from several sparse angles;

[0009] Select the target tissue to be modeled on the 2D image; use a sparse reconstruction algorithm to perform 3D virtual modeling on the selected target tissue to form a 3D modeling image of the target tissue;

[0010] Register the 3D modeling image with the 2D image, and calculate the first transformation relationship between the visual coordinate system and the motion coordinate system of the imaging device;

[0011] Adjust the initial pose of the 3D modeling image, and calculate the target pose of the imaging device corresponding to the adjusted pose based on the first transformation relationship and the adjusted pose of the 3D modeling image; Control the imaging device to reach the target pose to obtain the expected imaging perspective of the target tissue.

[0012] In some embodiments, the method for guiding the pose of an imaging device based on sparse reconstruction further includes:

[0013] Identify the surgical instrument, and align and superimpose the pose of the surgical instrument with the visual coordinate system;

[0014] Based on the preset path planning, automatically control the imaging device to dynamically adjust its pose to acquire tomographic images of the target tissue at the expected imaging perspective;

[0015] Identify the relative position relationship between the target tissue and the surgical instrument in the tomographic image; Issue a warning when the surgical instrument deviates from the preset path planning for timely correction.

[0016] In some embodiments, the imaging device has a multi-angle imaging function;

[0017] Before automatically acquiring 2D images of a simulated patient's specified area from a number of sparse angles using the imaging device, it further includes: presetting the sparse angles and the number of images acquired by the imaging device; making the acquired 2D images have sufficient information to support the sparse reconstruction of the target tissue.

[0018] In some embodiments, automatically acquiring 2D images of a simulated patient's specified area from a number of sparse angles using the imaging device specifically includes:

[0019] The imaging device is a fixed table combined with a movable C-arm imaging device; The simulated patient is placed on the fixed table; Image acquisition is performed at the preset sparse angles by the movable C-arm;

[0020] Or, the imaging device is an imaging device with a rotatable table combined with a fixed radiation source / detector; The simulated patient is placed on the rotatable table; Image acquisition is performed at the preset sparse angles by rotating the rotatable table.

[0021] In some embodiments, performing 3D virtual modeling on the selected target tissue using a sparse reconstruction algorithm; Specifically includes:

[0022] Through a generative adversarial network, candidate data of the target tissue can be generated by a generator in iterative reconstruction;

[0023] Then, it is compared with the 2D image, and feature data is extracted from the 2D image of the sparse view; the feature data is decoded to predict the intensity of three-dimensional points; the reconstruction is guided to be more realistic based on the intensity predicted by the three-dimensional points.

[0024] In some embodiments, for registering the 3D modeling image with the 2D image, a first conversion relationship between the visual coordinate system and the motion coordinate system of the imaging device is calculated; specifically including:

[0025] A second conversion relationship between the imaging coordinate system and the motion coordinate system is pre-calibrated;

[0026] The 3D modeling image is registered with the 2D image, a degree-of-freedom conversion matrix generated during the registration is calculated, and the inverse matrix of the degree-of-freedom conversion matrix is obtained to acquire a third conversion relationship between the imaging coordinate system and the visual coordinate system;

[0027] Based on the second conversion relationship and the third conversion relationship, the first conversion relationship is calculated.

[0028] In some embodiments, for registering the 3D modeling image with the 2D image; specifically including:

[0029] For the 3D modeling image, its corresponding 2D projection image is calculated by an orthographic projection method;

[0030] The 2D projection image is rigidly registered with the 2D image; the target tissue is marked on the registered 2D image.

[0031] In some embodiments, for registering the 3D modeling image with the 3D standard model of the target tissue, a first conversion relationship between the visual coordinate system and the motion coordinate system of the imaging device is calculated; specifically including:

[0032] The 3D standard model and the motion coordinate system are pre-calibrated;

[0033] Feature extraction and feature matching are respectively performed on the 3D modeling image and the 3D standard model, and a transformation matrix between the two is solved;

[0034] Based on the pre-calibration result of the 3D standard model and the transformation matrix, the first conversion relationship is calculated.

[0035] Second aspect, the present application also discloses an imaging device positioning guiding system based on sparse reconstruction, which is used to implement the steps of the method described in any of the above embodiments; including:

[0036] An image acquisition module, which is used to use an imaging device to automatically acquire 2D images of a specified area of a simulated patient from a number of sparse angles;

[0037] A selection module, which is used to select a target tissue to be modeled on the 2D image;

[0038] A 3D reconstruction module, which is used to perform 3D virtual modeling on the selected target tissue by using a sparse reconstruction algorithm to form a 3D modeled image of the target tissue;

[0039] A registration module, which is used to register the 3D modeled image with the 2D image and calculate a first conversion relationship between the visual coordinate system and the motion coordinate system of the imaging device;

[0040] Or, the registration module is used to register the 3D modeled image with a 3D standard model of the target tissue and calculate a first conversion relationship between the visual coordinate system and the motion coordinate system of the imaging device;

[0041] A pose adjustment module, which is used to adjust the initial pose of the 3D modeled image, and calculate the target positioning of the imaging device corresponding to the adjusted pose based on the first conversion relationship and the adjusted pose of the 3D modeled image; control the imaging device to reach the target positioning to obtain an expected imaging perspective of the target tissue.

[0042] In some embodiments, it further includes: an instrument identification module, which is used to identify surgical instruments and align and superimpose the poses of the surgical instruments with the visual coordinate system;

[0043] The pose adjustment module is further used to automatically control the imaging device to dynamically adjust its positioning based on a preset path planning to acquire tomographic images of the target tissue at the expected imaging perspective;

[0044] A warning and navigation module, which is used to identify the relative position relationship between the target tissue and the surgical instrument in the tomographic image; issue a warning when the surgical instrument deviates from the preset path planning for timely correction.

[0045] In some embodiments, the imaging device positioning guiding system based on sparse reconstruction further includes: a setting module, which is used to preset the sparse angles and quantities of images acquired by the imaging device; make the acquired 2D images have sufficient information to support the sparse reconstruction of the target tissue.

[0046] In some embodiments, the 3D reconstruction module is specifically configured to: through a generative adversarial network, generate candidate volume data of the target tissue by using a generator in iterative reconstruction; then compare it with the 2D image, and extract feature data from the 2D image of the sparse view; decode the feature data to predict the intensity of three-dimensional points; and guide the reconstruction to be more realistic based on the intensity predicted by the three-dimensional points.

[0047] In some embodiments, the registration module further includes a coordinate system conversion sub-module, which is configured to pre-calibrate the second conversion relationship between the imaging coordinate system and the motion coordinate system; register the 3D modeling image with the 2D image, calculate the degree-of-freedom conversion matrix generated during the registration process, and obtain the inverse matrix of the degree-of-freedom conversion matrix to acquire the third conversion relationship between the imaging coordinate system and the visual coordinate system; and calculate the first conversion relationship based on the second conversion relationship and the third conversion relationship.

[0048] In some embodiments, the registration module further includes a 2D-3D registration sub-module, which is specifically configured to: for the 3D modeling image, calculate its corresponding 2D projection image by means of orthographic projection; perform rigid registration on the 2D projection image and the 2D image; and label the target tissue on the registered 2D image.

[0049] In a third aspect, the present application also discloses an imaging device, which includes the imaging device positioning and guiding system based on sparse reconstruction described in any of the above embodiments.

[0050] Compared with the prior art, the present application has at least one of the following beneficial effects:

[0051] 1. The imaging device positioning and guiding method of the present application realizes the accurate positioning of the patient's target tissue and the guiding of the imaging device positioning through technologies such as image registration and coordinate system conversion, in combination with the target position of the surgical plan. The automatic guidance can not only automatically adjust the position and angle of the imaging device according to the preoperative plan or real-time image to ensure that the imaging perspective is always in the best state, thereby improving the accuracy of the surgery, but also the automatic guidance system can quickly adjust the imaging device to the preset position without manual adjustment and repeated confirmation, significantly shortening the operation time and further reducing radiation exposure. It reduces the dependence on the experience of the operator, enabling the surgical team to use the imaging device more easily. It improves the digital and intelligent level of the operating room.

[0052] 2. Based on the three-dimensional model, the present application can pre-plan the intervention path of surgical instruments (such as bone nails or guide pins), align the pose of the surgical instruments with the three-dimensional model coordinate system, and issue a warning in real time or display correction data if it deviates from the established trajectory. By docking with an external surgical navigation system, the planned surgical route can be mapped to the current bone position of the patient, the planned path can be real-time overlapped with the patient's three-dimensional model, and the imaging perspective can be dynamically adjusted, providing intuitive intraoperative navigation for doctors.

[0053] 3. Combine the imaging device positioning guiding method and path planning of the present application. The imaging device positioning guiding system of the present application automatically adjusts the perspective of the imaging device based on a preset path, can achieve real-time precise positioning and path correction of surgical instruments, and significantly improves the safety and efficiency of surgery. Through dynamic optimization of the imaging angle, the automatic correction function can timely correct the deviation of the instrument, ensure that the surgical field always focuses on the key operation area, and reduce the operation burden of doctors for frequently adjusting the device. Make the complex surgical process more standardized and controllable, providing key technical support for precision medicine.

[0054] 4. The solution of guiding the imaging device through the three-dimensional reconstruction and coordinate system conversion technology of low-dose sparse shooting of the present application can complete high-quality reconstruction with a very small amount of sampling data, significantly reduce the need for data acquisition, and thus reduce the radiation dose. The sparse reconstruction technology reduces the consumption of computing resources and improves the reconstruction efficiency through sparse representation and optimization algorithms. Through optimization algorithms and sparse representation, high-quality reconstruction can be completed in a short time. Based on this technology, the ray dose can be significantly reduced, the scanning time can be shortened, the efficiency of intraoperative or preoperative-intraoperative linkage can be improved, and it has important application value in the fields of spinal internal fixation, joint replacement, fracture reduction, and interventional radiology catheter operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The above characteristics, technical features, advantages and their implementation manners of the present application will be further described below in a clear and understandable manner in combination with the drawings and preferred embodiments.

[0056] Figure 1 is a flowchart of the steps of a method for guiding the positioning of an imaging device based on sparse reconstruction according to the present application;

[0057] Figure 2 is a flowchart of the steps of an embodiment of a method for guiding the positioning of an imaging device based on sparse reconstruction according to the present application;

[0058] Figure 3 is a flowchart of the steps of another embodiment of a method for guiding the positioning of an imaging device based on sparse reconstruction according to the present application;

[0059] Figure 4This is a flowchart of the steps of another embodiment of a method for guiding the positioning of an imaging device based on sparse reconstruction according to the present application;

[0060] Figure 5 This is a structural block diagram of an embodiment of a system for guiding the positioning of an imaging device based on sparse reconstruction according to the present application. Detailed implementation manners

[0061] In the following description, for the purpose of illustration rather than limitation, specific details such as specific system structures and technologies are presented to thoroughly understand the embodiments of the present application. However, those skilled in the art should clearly understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present application.

[0062] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or groups.

[0063] For the sake of simplicity of the drawings, only the parts related to the invention are schematically shown in each figure, and they do not represent the actual structure of the product. Additionally, for the sake of simplicity and easy understanding of the drawings, in some figures, components with the same structure or function are only schematically shown for one of them, or only one of them is marked. In this article, "one" not only means "only this one", but also can mean "more than one" situation.

[0064] It should also be further understood that the term "and / or" used in the specification and appended claims of the present application refers to any combination and all possible combinations of one or more of the related listed items, and includes these combinations.

[0065] In this article, it should be noted that unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to specific situations.

[0066] In addition, in the description of the present application, the terms "first", "second", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0067] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will describe the specific implementation manners of the present application with reference to the accompanying drawings. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained according to these drawings, and other implementation manners can also be obtained.

[0068] In the fields of clinical orthopedics, neurosurgery, and interventional radiology, etc., accurate three-dimensional image information is of great significance for surgical planning, instrument guidance, and intraoperative navigation.

[0069] In the prior art, conventional three-dimensional image information needs to be acquired by CT or CBCT rotating one week. For example, usually, the X-ray device needs to perform large-angle continuous scanning (such as 360°) around the patient or within a certain angular range, and use the classic FDK algorithm (Feldkamp-Davis-Kress) or the convolution back-projection algorithm for three-dimensional reconstruction. This method can obtain relatively accurate three-dimensional volume data, but there are problems such as high radiation dose, long scanning time, and complex device movement.

[0070] In the prior art, some advanced operating rooms are equipped with real-time CT, MR, or 3D C-arm devices to obtain accurate body position data for imaging device positioning guidance. However, such devices are often expensive and have high requirements for the operating room environment, and are not applicable to all hospitals and departments.

[0071] Currently, the guidance for imaging device positioning often relies on the doctor's experience judgment for intraoperative clinical operations. On the one hand, after the patient is placed on the table, it is necessary to locate the spinal segment for surgery. Usually, it is necessary to perform multiple fluoroscopies from head to foot or from foot to head, and then locate the vertebra to be operated on. This operation method has low requirements for doctors, but it increases the surgical radiation dose and surgical time. On the second hand, after the positioning needle or screw is inserted, it is necessary to perform multiple anteroposterior and lateral fluoroscopies to check the entry point angle and depth, and whether the implantation path is located within the pedicle channel. This operation highly depends on the doctor's experience (forming a three-dimensional image in the doctor's mind through two-dimensional images), and then judging the screw placement position. Moreover, there are standard positioning requirements for the imaging angle or position of two-dimensional images clinically, which also increases the doctor's learning difficulty and the surgical time prolongation caused by repeated positioning.

[0072] In summary, the above operations will all result in increased radiation and extended operation time. Some operations also highly rely on the doctor's experience and subjective judgment, making it difficult to standardize, and it requires a long-term accumulation of surgical cases to achieve better clinical judgment. Therefore, to reduce the radiation dose received by patients, shorten the acquisition time, and reduce the volume and cost of the equipment, this application proposes a technical solution that only needs to acquire X-ray projection images at a small number of angles (such as 2 to 10 angles), and on this basis, quickly reconstruct the three-dimensional information of the target bone to guide the positioning of the imaging device. Refer to Figure 1 , through technologies such as image registration and coordinate system conversion, combined with the target position in the surgical plan, this solution realizes the precise positioning of the patient's target tissue and the guidance of the imaging device positioning. It has significant clinical value and economic value in orthopedic and interventional radiology surgeries.

[0073] Refer to the attached Figure 2 As shown, an embodiment of a method for guiding the positioning of an imaging device based on sparse reconstruction in this application specifically includes the following steps:

[0074] S100, use the imaging device to automatically acquire 2D images of the specified area of the simulated patient from a number of sparse angles.

[0075] Specifically, the imaging device includes a CT device, a CBCT device, an X-ray device, or other existing medical imaging devices. The imaging device has a multi-angle imaging function.

[0076] This application only needs to take X-ray projections or CBCT images at a number of different sparse angles, reducing the high-dose acquisition of a full rotation (360°) in the traditional method. Each angle can be scanned by a mobile C-arm, a suspended X-ray machine, or a bedside CBCT device.

[0077] S200, select the target tissue to be modeled on the 2D image.

[0078] Specifically, for the 2D images acquired above, select the organizational structures to be modeled respectively. Manual drawing or automatic segmentation algorithms can be used for the possible 2D image processing and bone segmentation methods.

[0079] S300, use a sparse reconstruction algorithm to perform 3D virtual modeling on the selected target tissue to form a 3D modeling image of the target tissue.

[0080] Specifically, if the organizational structure is manually depicted in the previous step, 3D modeling is performed according to the depicted organizational structure. If a region of interest in the image is defined, the organizational structures or implants within the region are automatically identified for 3D modeling to form a 3D modeling image, which allows the user to intuitively view the organizational structures and implants in 3D form.

[0081] When the projection angles are much fewer than the number required by traditional reconstruction algorithms (such as the FDK algorithm), directly using classical algorithms often leads to reconstruction distortion or extremely high noise. Therefore, extremely sparse reconstruction algorithms can be adopted to reconstruct the three-dimensional model of the target bone. Most of these algorithms originate from the ideas of Compressed Sensing and sparse constraints.

[0082] S400, register the 3D modeling image with the 2D image, and calculate the first conversion relationship between the visual coordinate system and the motion coordinate system of the imaging device.

[0083] Specifically, calculate the conversion relationship of the coordinate system from the registration process.

[0084] S500, adjust the initial pose of the 3D modeling image, and based on the first conversion relationship and the adjusted pose of the 3D modeling image, calculate the target pose of the imaging device corresponding to the adjusted pose, and control the imaging device to reach the target pose to obtain the expected imaging perspective of the target tissue.

[0085] Specifically, the 3D modeling image can be adjusted, such as operations like rotation, translation, and zooming. The system automatically converts the visual coordinate system into the motion coordinate system of the C-arm, and can accurately prompt the target position of the device pose. The user can move to the target pose manually or automatically according to the prompt.

[0086] Specifically, the automatic guidance system can quickly adjust the imaging angle and parameters according to the surgical needs, reducing the time and effort required for manual adjustment of the C-arm, and can significantly shorten the surgical time. By optimizing the imaging parameters and reducing unnecessary adjustments, the radiation dose of patients and medical staff can be reduced. The automatic guidance system can automatically adjust the position and angle of the C-arm according to the preoperative plan or real-time images, ensuring that the imaging perspective is always in the best state, thereby improving the accuracy of the surgery.

[0087] Refer to the attached Figure 3 As shown in the specification, another embodiment of an imaging device pose guidance method based on sparse reconstruction provided by the present application specifically includes the following steps:

[0088] S300, perform sparse reconstruction on the target tissue to obtain a 3D modeling image.

[0089] S400, register the 3D modeling image with the 3D standard model of the target tissue, and calculate the first conversion relationship between the visual coordinate system and the motion coordinate system of the imaging device;

[0090] S500 adjusts the initial pose of the 3D modeling image, calculates the target pose of the imaging device corresponding to the adjusted pose of the 3D modeling image based on the first conversion relationship and the adjusted pose of the 3D modeling image, and controls the imaging device to reach the target pose to obtain the expected imaging view of the target tissue.

[0091] Specifically, 3D-3D registration aligns two three-dimensional data sets through spatial transformation. The core process includes: first, extracting point cloud or voxel features, establishing corresponding relationships through feature matching or intensity similarity measurement; then, using an optimization algorithm to solve the optimal rigid / non-rigid transformation matrix (including rotation, translation, and deformation parameters); finally, completing spatial alignment through resampling.

[0092] The target tissue is, for example, bone, and its 3D standard model is a pre-constructed bone standard model. In some embodiments, one source of the bone standard model is to statistically analyze and cluster the historically collected bone data. During clustering, according to the division of different genders and age groups, such as male aged 10 - 12, female aged 40 - 45, male aged 80 - 85, etc., a corresponding standard bone model is generated based on statistics. In other embodiments, a sparse reconstruction algorithm based on an artificial intelligence model is used to generate the bone standard model. First, a large amount of bone data scanned by CT / MRI is collected as a training set, and the bone morphological features are learned through a deep learning network (such as 3DCNN or generative adversarial network); then, a sparse sampling strategy is constructed to reduce the amount of input data using the theory of compressed sensing; during the training process, the network is optimized by combining image prior knowledge and physical constraints so that the model can reconstruct the standard bone model from the sparse sampling data. The algorithms for standard model reconstruction have made certain progress in the prior art, and this application will not elaborate too much on them.

[0093] The standard model provides an accurate anatomical reference framework for surgical planning, enabling doctors to simulate the instrument path preoperatively and predict the positions of key structures, thereby improving the predictability of complex orthopedic and spinal surgeries. During the operation, through the registration and fusion of real-time images and the standard model, rapid registration of the intraoperative image and the motion coordinate system of the imaging device can be achieved. Based on the standard model, personalized surgical plans can be formulated, significantly reducing the number of intraoperative fluoroscopies. The standard model also serves as a reference benchmark for instrument navigation. The preset navigation path is superimposed on the three-dimensional model. When the instrument deviates from the planned path, an alarm is automatically triggered, and the projection angle of devices such as the C-arm is dynamically adjusted through multi-modal image fusion technology to ensure the best imaging view is always obtained.

[0094] Refer to the attached Figure 4 As shown in the specification, another embodiment of an imaging device pose guiding method based on sparse reconstruction is provided in this application. Based on any of the above embodiments of the method, it further includes:

[0095] S600 identifies the surgical instruments and aligns and superimposes the poses of the surgical instruments with the visual coordinate system.

[0096] S700 automatically controls the imaging device to dynamically adjust its position to acquire tomographic images of the target tissue from the expected imaging perspectives based on a preset path plan.

[0097] S800 identifies the relative positional relationship between the target tissue and the surgical instruments in the tomographic images; issues a warning when the surgical instruments deviate from the preset path plan for timely correction.

[0098] Specifically, in addition to the functions of accurately positioning the patient's target tissue and guiding the positioning of the imaging device as described above, other functions of the imaging device positioning and guiding method of the present application also include: surgical instrument navigation: users can judge the relative positional relationship between the tissue structure and the implant through the tomographic images of 3D modeling, and further evaluate the surgical effect.

[0099] Specifically, taking nail path planning and bone nail navigation as an example, the nail path (such as the pedicle screw channel during spinal internal fixation) can be pre-planned in the three-dimensional model, and the center line of the nail path is merged and displayed with the three-dimensional model of the bone. Real-time monitoring of the insertion angle and depth of the bone nail: Align the pose of the bone nail (or guide pin) with the three-dimensional model coordinate system, and issue a warning in real time or display correction data if it deviates from the established trajectory. By docking with an external surgical navigation system, the planned surgical route can be mapped to the current bone position of the patient to provide intraoperative guidance for doctors.

[0100] Navigation of other instruments: For example, the bending and alignment of the steel plate for fracture internal fixation, the accurate grinding angle of joint replacement instruments, the navigation path of the interventional catheter in the blood vessel, etc., can all achieve positioning and deviation correction by means of the same set of coordinate transformation frameworks.

[0101] Combined with the imaging device positioning and guiding method and path planning of the present application. The imaging device positioning and guiding system of the present application automatically adjusts the perspective of the imaging device based on a preset path, can achieve real-time precise positioning of surgical instruments and path deviation correction, and significantly improves the safety and efficiency of surgery. The system dynamically optimizes the imaging angle, and the automatic deviation correction function can timely correct the instrument deviation, ensuring that the surgical field always focuses on the key operation area, reducing the operation burden of doctors for frequent equipment adjustment. Making complex surgical procedures more standardized and controllable, providing key technical support for precision medicine.

[0102] In another implementation of this embodiment, based on the same principle, the position of a surgical instrument (such as a bone nail) can also be constructed in a three-dimensional space, and its relative position relationship with the three-dimensional model of the target tissue (such as the target bone) can be compared to determine whether the nail insertion angle is accurate and whether the depth is excessive, thereby achieving a certain degree of surgical navigation. The planned nail path and surgical route can also be compared with the current instrument position to generate a spatial transformation matrix to guide the surgery.

[0103] Based on the above embodiment, another embodiment of the imaging device positioning and guiding method based on sparse reconstruction disclosed in this application further includes, before step S100: presetting the sparse angles and quantities of the images collected by the imaging device. This enables the collected 2D images to have sufficient information to support the sparse reconstruction of the target tissue.

[0104] Specifically, start the automatic multi-angle photography mode, and the device (imaging device) automatically collects 2D images at preset angles (usually the anteroposterior and lateral views), and the number of collected 2D images is generally 2 - 10. These projections are far fewer in number than the requirements for conventional three-dimensional reconstruction, but through subsequent special sparse reconstruction techniques and registration strategies, a three-dimensional model that meets the navigation requirements can still be obtained.

[0105] Specifically, multi-angle imaging is achieved through the following sub-steps: In one implementation of this embodiment, the imaging device is a fixed table combined with a movable C-arm imaging device. Simulate the patient placed on the fixed table, and the movable C-arm is used to collect images at preset sparse angles.

[0106] Specifically, a movable C-arm imaging device, such as a C-arm X-ray machine, mainly consists of the following parts: C-shaped frame: Connects the X-ray generator and the detector, shaped like the letter "C", facilitating flexible movement during surgery. X-ray generator: Used to generate X-rays. Detector: Usually includes an image intensifier and a CCD camera, used to collect X-ray images. Image processing system: Used to process and display the imaging results. Fixed table + movable C-arm: Keeps the patient relatively stationary, and the C-arm is used to project and take pictures at several key angles (such as the anteroposterior, lateral, or oblique views).

[0107] In another implementation of this embodiment, the imaging device is an imaging device with a rotatable table combined with a fixed X-ray source / detector. Simulate the patient placed on the rotatable table, and the rotatable table is rotated to collect images at preset sparse angles.

[0108] Specifically, rotatable table + fixed X-ray source / detector: Rotate the table by a small angle, and perform a projection acquisition each time it rotates to a specified angle.

[0109] Based on the above embodiments, another embodiment of a method for guiding the positioning of an imaging device based on sparse reconstruction is disclosed in the present application. The step S200: Select the target tissue to be modeled on the 2D image; specifically includes:

[0110] S210, Manually depict the organizational structure or implant of interest.

[0111] Specifically, manually selecting the region of interest is the most direct method. The user can manually draw a rectangle, polygon or other shape to select the target area through image processing software or special tools. For example: Use the irregular polygon tool for semi-automatic annotation, and precisely outline the region of interest by drawing with the left mouse button and erasing with the right mouse button.

[0112] S220, Depict the organizational structure or implant of interest by automatically delimiting the ROI area. To obtain more accurate results; by defining the image area of interest.

[0113] Specifically, automatically selecting the region of interest usually requires the aid of image processing algorithms or deep learning models. For example: Method based on object detection algorithm: Use a pre-trained object detection model (such as YOLO, SSD, etc.) to identify the target tissue and select the area. For example, use OpenCV to load the object detector, run the detection on the image, and determine whether the target is within the preset ROI area.

[0114] Method based on deep learning segmentation model: Use segmentation models such as U-Net to automatically identify and segment the target tissue. These models can precisely select the region of interest by learning image features.

[0115] At the same time, more accurate results can also be obtained by delimiting the ROI area.

[0116] Based on the above embodiments, another embodiment of a method for guiding the positioning of an imaging device based on sparse reconstruction is disclosed in the present application. The step S300 uses a sparse reconstruction algorithm to perform 3D virtual modeling on the selected target tissue to form a 3D modeling image of the target tissue. Specifically includes:

[0117] S311, Through a generative adversarial network, the candidate volume data of the target tissue can be generated by the generator in iterative reconstruction.

[0118] S312, Then compare it with the 2D image, extract feature data from the 2D image of the sparse view; decode the feature data to predict the intensity of the three-dimensional points; guide the reconstruction to be more realistic based on the intensity predicted by the three-dimensional points.

[0119] Specifically, the extremely sparse reconstruction technology of this application can reconstruct three-dimensional volumetric data using only a few X-ray projections taken at different angles. The sparse reconstruction projection algorithm is used to reconstruct the target bone.

[0120] In one implementation of this embodiment, extremely sparse reconstruction is achieved by combining the traditional reconstruction method with a generative network. That is, extremely sparse reconstruction is achieved based on the method of regularization and iterative optimization.

[0121] Specifically, in S321, with the help of the NeRF (Neural Radiance Fields) algorithm. At extremely few viewpoints, by fitting the implicit representation of the neural radiance field to the rendering equation, the three-dimensional structure of the target bone can be reconstructed.

[0122] In S322, with the help of the DDPM algorithm: first, noise diffusion is performed on the three-dimensional bone data, and then the reverse denoising process is learned; when there are only a small number of projections, through feature extraction and adversarial learning, the three-dimensional voxels or implicit representations are optimized. Or, with the help of the GAN algorithm: the input of the generator is the information feature map of a small number of angular projections, and the output is the predicted three-dimensional model; the discriminator is responsible for evaluating the difference between the predicted model and the real three-dimensional model or real projections.

[0123] When the requirement for the fineness of soft tissues is not high (mainly focusing on bone information), this type of method can often achieve good enough accuracy. Even if the number of projections is insufficient or the angles are incomplete, good reconstruction results can be obtained with the help of a large amount of prior learning.

[0124] In these methods, in order to further improve the reconstruction quality, existing prior information (such as bone shape prior, anatomical structure prior) is often combined, and special shape model constraints can be imposed on the bones.

[0125] Preferably, it further includes S323, with the help of the Ttal Variatin (TV) regularization algorithm, by constraining the image gradient, suppressing noise while maintaining the structural edges, such as common iterative algorithms like SART (Simultaneus AlgebraicRecnstructin Technique)+TV, S-SART+TV, etc. L1 or approximate L0 sparse constraints are imposed in the wavelet domain or dictionary domain to reduce artifacts.

[0126] In some other implementations of this embodiment, a generative model is trained with a large amount of three-dimensional data such as CT / MR / CBCT and corresponding simulated projections (or real multi-angle projections), so that it can infer the target three-dimensional structure based on a small number of projections during the inference stage. That is, through a large number of 2D-3D data pairs, or simulated data, a spatial three-dimensional model is reconstructed from several two-dimensional projections at different angles.

[0127] The training data can be obtained from public medical image databases (such as SpineWeb, VerSe, etc.) or digital models simulating the human skeleton (Mesh model / CT data).

[0128] Taking the GAN and DDPM algorithms as examples: S331, by means of the DDPM (Denising Diffusin PrbabilisticMdels) algorithm. Through the learning of a large amount of 2D-3D aligned data, missing information can be complemented under extremely sparse projections to generate relatively accurate three-dimensional volume data.

[0129] S332, by means of the GAN (Generative Adversarial Netwrks) algorithm. Through the generative adversarial network, candidate volume data can be generated by the generator in iterative reconstruction, and then compared with the real projection, and the discriminator is used to guide the reconstruction to be more realistic.

[0130] In some other embodiments of this embodiment, when high-quality three-dimensional reconstruction cannot be directly obtained or the algorithm complexity is expected to be reduced, a 2D-3D registration strategy can be adopted. The specific method is as follows:

[0131] S341, establish a standard three-dimensional bone model. Specifically, use the pre-constructed or obtained from the database standard three-dimensional model of the human skeleton, or construct a relatively accurate reference model according to the early full CT data of the same person.

[0132] S342, 2D projection matching. Specifically, starting from the standard model, change its pose in space (6 degrees of freedom: three-dimensional translation + three-dimensional rotation), and scale change or local deformation can be added (1 to several additional degrees of freedom can be added), and then simulate X-ray projections at different angles. Compare the simulated projection with the real X-ray projection, and use the similarity (or difference) as an index to continuously adjust the pose / deformation parameters of the standard model to minimize the gap.

[0133] S343, search algorithm. Specifically, common search algorithms can be used, such as: Pwell, Nelder–Mead, evolutionary algorithms (Genetic / Evlutinary Algrithms), etc., which are search strategies commonly used for multi-dimensional continuous optimization. Through these search methods, continuously adjust the registration parameters to obtain the state where the difference between the simulated projection and the real projection is the smallest, and then the position and shape of the target bone in space can be obtained.

[0134] S344, calculate similarity or loss function. For example: use NCC (Nrmalized Crss-Crrelatin): normalize the cross-correlation of the projection image; or use MSE (Mean Squared Errr), MAE (Mean Abslute Errr): measure pixel-level differences; MI (Mutual Infrmatin): can measure nonlinear intensity distribution differences; SSIM (Structural Similarity Index): measure structural similarity; in practical applications, the optimal indicator can be selected according to the image characteristics, and even deep learning features can be combined for similarity calculation.

[0135] S345, bone segmentation and feature alignment. The captured 2D X-ray image can be segmented automatically or manually to obtain the bone contour; RI area selection or key point annotation (such as vertebral center, articular surface, upper edge of ilium, etc.) can help further improve the registration accuracy; the centroid, major and minor axes of the 3D model can be aligned with the corresponding features in the 2D projection to achieve fast initial pose estimation.

[0136] For example, we can use a standard 3D human skeleton model to perform 2D-3D comprehensive registration of our 2D X-ray images taken at multiple angles.

[0137] By changing the 6 degrees of freedom of the 3D model in space, plus scaling and some deformation, we can use the projection algorithm to generate different simulated X-ray projections at different angles. By comparing the difference between the simulated X-ray projection and the real X-ray projection, we can use the machine learning search algorithm to adjust and scale the standard model to the current position of the actual human skeleton. At this time, the difference between the projection of this model and the projection of the real human skeleton X-ray is the smallest. Here are several search strategies such as Powewell, Nelder-Mead, evolution, etc. At the same time, the difference between the simulated projection and the real X-ray can also be calculated using different loss functions, such as the NCC algorithm.

[0138] In particular, we can convert the 3D standard model into a point cloud to obtain features such as the center of mass and major and minor axes of the skeleton, and then quickly initialize the spatial position of the 3D model by aligning the features in 3D and 2D to points or quantities based on the center of mass, major and minor axes and other features on the 2D projection.

[0139] Preferably, in this embodiment, according to different clinical needs and equipment configurations, extremely sparse reconstruction and 2D-3D registration strategies can be used simultaneously or separately. For example:

[0140] First, 2D-3D registration is used to obtain the general position of the bones, and then an extremely sparse reconstruction algorithm is used to correct the details.

[0141] Alternatively, directly use an AI generation model to obtain a 3D result through a small number of projection reconstructions, and then perform rigid body registration based on a small number of marker points to reduce the computational amount.

[0142] Based on the above embodiments, another embodiment of an imaging device positioning guidance method based on sparse reconstruction is disclosed in the present application. In step S400: register the 3D modeling image with the 2D image, and calculate a first conversion relationship between the visual coordinate system and the motion coordinate system of the imaging device; specifically, it includes the following sub-steps:

[0143] S411, calibrate a second conversion relationship between the imaging coordinate system and the motion coordinate system.

[0144] S412, register the 3D modeling image with the 2D image, calculate a degree-of-freedom conversion matrix generated during the registration process, and obtain the inverse matrix of the degree-of-freedom conversion matrix to obtain a third conversion relationship between the imaging coordinate system and the visual coordinate system.

[0145] S413, calculate the first conversion relationship based on the second conversion relationship and the third conversion relationship.

[0146] Specifically, the motion coordinate system refers to a coordinate system centered on the motion origin of an imaging device with multi-angle imaging capabilities during angle transformation, and is used to describe the spatial pose of the imaging device, including positioning. Taking a C-arm X-ray device as an example, the motion coordinate system is a coordinate system with the C-arm motion center as the origin.

[0147] The imaging coordinate system refers to a coordinate system with the imaging center of the imaging device as the origin; taking a C-arm X-ray device as an example, the imaging coordinate system has the optical center of the C-arm X-ray device as the origin, the optical axis as the Z axis, and the horizontal and vertical coordinate axes of the image plane as the X axis and the Y axis.

[0148] The visual coordinate system refers to a coordinate system with the center of the target tissue as the origin, and is used to describe the spatial position of the target tissue.

[0149] Preferably, in this embodiment, in the imaging device, the three-dimensional coordinates of multiple target points in the visual coordinate system are obtained through an optical tracking device, and the two-dimensional coordinates of these target points in the imaging coordinate system are recorded simultaneously. Assuming that the coordinates of a certain point in space in the visual coordinate system are (X, Y, Z), and the coordinates in the imaging coordinate system are (x, y), the conversion relationship between the two can be expressed as a mathematical model in matrix form. According to the mathematical model, using the three-dimensional coordinates and two-dimensional coordinates of multiple target points, the parameters in the conversion matrix are solved by the least squares method or other optimization methods. Through the obtained matrix, the point (x, y, z) in the pixel coordinate system can be converted to the point (X, Y, Z) in the visual coordinate system, thereby realizing the conversion between the imaging coordinate system and the visual coordinate system.

[0150] If the CBCT device or C-arm device in the operating room has been geometrically calibrated, the device can be automatically adjusted according to the transformation matrix, so as to quickly align with the planned fluoroscopy angle or scanning angle. Based on the second conversion relationship between the pre-calibrated imaging coordinate system and the motion coordinate system, according to the properties of coordinate system conversion, the first conversion relationship between the visual coordinate system and the motion coordinate system of the imaging device can be solved using the known conversion relationship.

[0151] On the basis of the above embodiment, another embodiment of the imaging device positioning guidance method based on sparse reconstruction is disclosed in the present application. The step S400: registering the 3D modeling image and the 2D image, and calculating the first conversion relationship between the visual coordinate system and the motion coordinate system of the imaging device; specifically including the following sub-steps:

[0152] S421. For the 3D modeling image, calculate its corresponding 2D projection image by the method of orthographic projection.

[0153] S422. Register the 2D projection image and the 2D image.

[0154] S423. Mark the target tissue on the registered image.

[0155] S424. Calibrate the second conversion relationship between the imaging coordinate system and the motion coordinate system.

[0156] S425. Based on the registration of the 2D projection image and the 2D image, calculate the degree-of-freedom conversion matrix generated during the registration process, and obtain the inverse matrix of the degree-of-freedom conversion matrix to obtain the third conversion relationship between the imaging coordinate system and the visual coordinate system.

[0157] S426. Based on the second conversion relationship and the third conversion relationship, calculate the first conversion relationship.

[0158] Specifically, for the 3D modeling image, a 2D projection image is calculated by means of orthographic projection, registered with the 2D image collected in the first step described above, and the name codes of the tissue structure or implant are identified, such as clinical anatomy codes.

[0159] Taking the target bone as the target tissue as an example: After the three-dimensional model of the target bone in space is successfully constructed or registered to the actual position (denoted as U, i.e., the visual coordinate system), the transformation and alignment can be carried out in combination with the "perfect position" (denoted as P, i.e., the imaging coordinate system) planned before surgery.

[0160] The perfect position P: Before surgery, the doctor places the standard bone model or the bone model generated from the patient's CT in a coordinate system considered to be "the optimal positioning" (such as the ideal C-arm angle or the alignment of the internal fixation nail path, etc.) in the planning software. This position may also include specific reference points (such as anatomical positioning marks) or the placement positions of instruments.

[0161] The current actual position U: The real-time spatial pose of the patient's bone at the surgical site obtained by the sparse reconstruction or registration method in step 1 of this application.

[0162] Obtain the transformation matrix G from U (i.e., the visual coordinate system) to P (i.e., the imaging coordinate system):

[0163] When using the 2D-3D registration method: Essentially, it is to transform the standard model from the "perfect position" through translation, rotation, scaling, etc., and match it to the real projection; the obtained registration transformation matrix T makes P×T = U; then the inverse matrix G of T = T^{-1} can return the position U to the position P.

[0164] When using other sparse reconstruction methods: Several key points (such as bone prominences, joint centers, etc.) can be selected on the three-dimensional bone model, and rigid body registration or corresponding degree-of-freedom registration is carried out with the corresponding points on the perfect position model; the transformation matrix G from U to P is obtained.

[0165] More preferably, in specific use, if the goal is to return the bone to a specific body position (such as the surgical nail insertion angle), the device or the patient can be adjusted by performing the transformation G; if it is to move the operating table or the robotic arm, the mechanical actuator is driven according to the parameters (translation, rotation, scaling) of the matrix G to align the actual position to the perfect position.

[0166] In another implementation manner of this embodiment, after we successfully construct the three-dimensional model of the target bone space, we know its current position clinically.

[0167] For navigation and positioning, we construct a perfect positioning model in the image space. This bone model is placed at the perfect position P considered by the doctor in the image coordinate system in advance according to medical knowledge before the operation. And the position of the actual three-dimensional bone model of the patient we obtained in the image space is U.

[0168] Calculate the transformation matrix G from U (i.e., the visual coordinate system) to P (i.e., the imaging coordinate system). We move and scale the standard model from the perfect position P to the current position U, which is the 2D-3D registration process. As long as we place the standard model at the perfect position P in advance, then after registration, the generated 6-X (specifically 6 degrees of freedom, 9 degrees of freedom, or X degrees of freedom, etc., depending on which transformation is used. If it is a rigid body + simple scaling, it is 7 degrees of freedom, and more complex deformations can be added, but the essence of the method remains unchanged). The degrees of freedom transformation matrix T makes PXT = U. Therefore, we only need to obtain the inverse matrix G of T, which can guide us on how to return from the current position U of the patient's bone to the perfect position P.

[0169] In another implementation manner of this embodiment, we can use the feature points on the three-dimensional bone to construct the 6-X degrees of freedom state coordinates U of the current U bone in space. At the same time, there are the same feature points and key points on the standard bone model to construct the coordinate P. Similarly, the transformation matrix G from U to P can be obtained.

[0170] After knowing G (i.e., the third transformation relationship), the pre-calibrated CBCT device can directly move according to the information provided by G (if this device has undergone geometric correction), so that the device will be moved to the best viewing position that was pre-desired. If it is other devices, a calibration can be performed first between P (i.e., the imaging coordinate system) and the CBCT device (i.e., the motion coordinate system).

[0171] Based on the above embodiments, another embodiment of the imaging device positioning guidance method based on sparse reconstruction disclosed in this application, the step S400: register the 3D modeling image with the 3D standard model of the target tissue, and calculate the first transformation relationship between the visual coordinate system and the motion coordinate system of the imaging device; specifically includes the following sub-steps:

[0172] S431, pre-calibrate the 3D standard model and the motion coordinate system.

[0173] S432, perform feature extraction and feature matching on the 3D modeling image and the 3D standard model respectively, and solve the transformation matrix between the two.

[0174] Preferably, other methods such as multi-point or point cloud registration are performed through 3D bone feature point recognition. A transformation matrix between the current spatial state of the sparse reconstruction model and the desired standard pose is found. Specifically, first, key anatomical landmark points such as joint centers and bony prominences, or surface point clouds of the 3D modeling image (reconstruction model) and the standard model are extracted; then, the iterative closest point (ICP) algorithm or feature matching methods (such as SIFT3D, FPFH, etc.) are used for rough registration to preliminarily align the two models; then, by optimizing the rigid / non-rigid transformation parameters (rotation matrix R and translation vector t), the corresponding point distance error (such as the Huber loss function) is minimized; finally, the optimal transformation matrix T (4x4 homogeneous matrix) is output, so that the spatial error between the current reconstruction model after being transformed by T and the standard pose is minimized.

[0175] S433, based on the pre-calibration result of the 3D standard model and the transformation matrix, the first transformation relationship is calculated.

[0176] Based on the above embodiments, another embodiment of the imaging device positioning guidance method based on sparse reconstruction disclosed in the present application, the step S500 specifically includes:

[0177] If the goal is to return the bone to a specific position (such as the surgical nail insertion angle), the device or the patient can be adjusted by performing the transformation G.

[0178] If it is to move the operating table or the robotic arm, the mechanical actuator is driven according to the parameters (translation, rotation, scaling) of the matrix G to align the actual position to the perfect position.

[0179] Specifically, the 3D modeling image can be adjusted, such as rotation, translation, zooming, etc. The system automatically converts the visual coordinate system into the motion coordinate system of the C-arm, and can accurately prompt the target position of the device positioning. The user can move to the target positioning position manually or automatically according to the prompt.

[0180] Some embodiments of the imaging device positioning guidance method based on sparse reconstruction provided in this embodiment: taking the scenarios that can be actually used in orthopedics or interventional radiology as examples:

[0181] 1. Spinal pedicle screw insertion: Before surgery, a standard spinal model or a personalized model constructed from the patient's CT is used to plan the screw insertion channel; during surgery, sparse reconstruction is performed using a small number of X-ray projections at angles, or 2D-3D registration is directly performed to obtain the true pose of the patient's vertebrae; the transformation matrix G is calculated to guide the movement of the operating table or the C-arm and adjust the angle of the screw insertion handle. The depth and direction of the screw insertion are evaluated in real time to ensure that it is far from the nerve and blood vessel structures.

[0182] 2. Pelvic fracture reduction: First, reset the pelvic model to the anatomical standard position or the ideal alignment considered by the doctor in the software; take 2 - 3 pelvic X-rays during the operation. After registration / reconstruction, the actual dislocation amount of the pelvis is known, and the reduction operation is guided according to the translation and rotation parameters prompted by G, reducing repeated radiography and radiation dose.

[0183] 3. Hip or knee joint replacement: Traditional joint replacement requires frequent intraoperative fluoroscopy or repeated comparison. After using this application, the joint spatial posture can be quickly obtained after several low-dose projections, and compared with the pre-planned prosthesis position, so as to perform navigation correction during the operation, improving accuracy and shortening the time.

[0184] 4. Interventional radiology catheter navigation: During interventional treatment, both the fluoroscopy angle and the catheter insertion position need to be continuously adjusted. By registering 2D - 3D, the standard model of the bone or vascular stent is aligned with the real patient position, and the relationship between the current catheter and the target position can be obtained in real time with only a small number of fluoroscopy images.

[0185] Based on the same concept, this application also discloses an imaging device positioning guiding system based on sparse reconstruction. The system is used to implement the steps described in any one of the above method embodiments. Specifically, in an embodiment of an imaging device positioning guiding system based on sparse reconstruction of this application, referring to the attached Figure 5 shown in the specification, specifically includes:

[0186] An image acquisition module, which is used to automatically acquire 2D images of the specified area of the simulated patient from several sparse angles using an imaging device.

[0187] A selection module, which is used to select the target tissue to be modeled on the 2D image.

[0188] A 3D reconstruction module, which is used to perform 3D virtual modeling on the selected target tissue using a sparse reconstruction algorithm to form a 3D modeling image of the target tissue.

[0189] A registration module, which is used to register the 3D modeling image with the 2D image and calculate the first conversion relationship between the visual coordinate system and the motion coordinate system of the imaging device.

[0190] Or, the registration module is used to register the 3D modeling image with the 3D standard model of the target tissue and calculate the first conversion relationship between the visual coordinate system and the motion coordinate system of the imaging device;

[0191] A pose adjustment module, which is used to adjust the initial pose of the 3D modeling image, and calculate the target positioning of the imaging device corresponding to the adjusted pose based on the first conversion relationship and the adjusted pose of the 3D modeling image. Control the imaging device to reach the target positioning to obtain the expected imaging perspective of the target tissue.

[0192] Based on the above embodiments, another embodiment of the imaging device positioning guidance system based on sparse reconstruction disclosed in the present application further includes: an instrument recognition module, configured to recognize surgical instruments and align and superimpose the poses of the surgical instruments with the visual coordinate system;

[0193] The pose adjustment module is further configured to automatically control the imaging device to dynamically adjust the position to acquire tomographic images of the target tissue at an expected imaging angle based on a preset path plan;

[0194] A warning navigation module, configured to recognize the relative position relationship between the target tissue and the surgical instrument in the tomographic image; issue a warning when the surgical instrument deviates from the preset path plan for timely correction.

[0195] Based on the above embodiments, another embodiment of the imaging device positioning guidance system based on sparse reconstruction disclosed in the present application, the imaging device positioning guidance system based on sparse reconstruction further includes: a setting module, configured to preset the sparse angles and quantities of images acquired by the imaging device, so that the acquired 2D images have sufficient information to support the sparse reconstruction of the target tissue.

[0196] Another embodiment of the imaging device positioning guidance system based on sparse reconstruction provided by the present application. Based on any one of the above embodiments of the system, the 3D reconstruction module is specifically configured to: through a generative adversarial network, generate candidate volume data of the target tissue by using a generator in iterative reconstruction, then compare it with the 2D image, extract feature data from the 2D image of the sparse view, decode the feature data to predict the intensity of three-dimensional points, and guide the reconstruction to be more realistic based on the intensity predicted by the three-dimensional points.

[0197] Another embodiment of the imaging device positioning guidance system based on sparse reconstruction provided by the present application. Based on any one of the above embodiments of the system, the registration module further includes a coordinate system conversion sub-module, configured to pre-calibrate the second conversion relationship between the imaging coordinate system and the motion coordinate system, register the 3D modeling image with the 2D image, calculate the degree-of-freedom conversion matrix generated during the registration process, obtain the inverse matrix of the degree-of-freedom conversion matrix to acquire the third conversion relationship between the imaging coordinate system and the visual coordinate system; calculate the first conversion relationship based on the second conversion relationship and the third conversion relationship.

[0198] In another implementation of this embodiment: The registration module further includes a 2D-3D registration sub-module, which is specifically configured to: calculate the corresponding 2D projection image of the 3D modeling image by means of orthographic projection; perform rigid registration on the 2D projection image and the 2D image; and mark the target tissue on the registered 2D image.

[0199] Based on the same concept, the present application also discloses an imaging device, which includes the imaging device positioning guidance system based on sparse reconstruction described in any one of the above embodiments; or the imaging device is used to implement the steps of the method for guiding the positioning of an imaging device based on sparse reconstruction described in any one of the above embodiments.

[0200] The imaging device has a multi-angle imaging function.

[0201] In one embodiment of an imaging device of the present application, the imaging device is a fixed table combined with a movable C-arm imaging device; a simulated patient is placed on the fixed table; image acquisition is performed through the movable C-arm at preset sparse angles;

[0202] In another embodiment of an imaging device of the present application, the imaging device is an imaging device with a rotatable table combined with a fixed radiation source / detector; a simulated patient is placed on the rotatable table; image acquisition is performed by rotating the rotatable table at preset sparse angles.

[0203] The method, system, and device for guiding the positioning of an imaging device based on sparse reconstruction of the present application have the same technical concept, and the technical details of the embodiments of the three can be mutually applicable. To avoid repetition, they will not be elaborated here.

[0204] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above division of each program module is used as an example. In practical applications, the above functions can be allocated to different program modules according to needs, that is, the internal structure of the device is divided into different program units or modules to complete all or part of the functions described above. Each program module in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one processing unit. The above integrated units can be implemented in the form of hardware or in the form of software program units. In addition, the specific names of each program module are only for the convenience of mutual distinction and do not limit the protection scope of the present application.

[0205] Although the preferred embodiments of the present application have been described, those skilled in the art can make additional changes and modifications once they know the basic creative concept. Therefore, the appended claims are intended to be construed as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

Claims

1. An imaging device positioning guiding method based on sparse reconstruction, characterized in that Including the following steps: Using an imaging device, automatically acquire 2D images of a specified area of a simulated patient from several sparse angles; Select the target tissue to be modeled on the 2D image; Adopt a sparse reconstruction algorithm to perform 3D virtual modeling on the selected target tissue to form a 3D modeling image of the target tissue; Register the 3D modeling image with the 2D image, and calculate the first conversion relationship between the visual coordinate system and the motion coordinate system of the imaging device; Alternatively, register the 3D modeling image with the 3D standard model of the target tissue, and calculate the first conversion relationship between the visual coordinate system and the motion coordinate system of the imaging device; Adjust the initial pose of the 3D modeling image, and based on the first conversion relationship and the adjusted pose of the 3D modeling image, calculate the target pose of the imaging device corresponding to the adjusted pose; control the imaging device to reach the target pose to obtain the expected imaging perspective of the target tissue.

2. The method for guiding the positioning of an imaging device based on sparse reconstruction according to claim 1, wherein It further includes: Identify the surgical instrument, and align and superimpose the pose of the surgical instrument with the visual coordinate system; Based on a preset path plan, automatically control the imaging device to dynamically adjust the pose to acquire tomographic images of the target tissue at the expected imaging perspective; Identify the relative position relationship between the target tissue and the surgical instrument in the tomographic image; issue a warning when the surgical instrument deviates from the preset path plan for timely correction.

3. The method for guiding the positioning of an imaging device based on sparse reconstruction according to claim 1, wherein The imaging device has a multi-angle imaging function; Before the step of using the imaging device to automatically acquire 2D images of a specified area of a simulated patient from several sparse angles, it further includes: presetting the sparse angles and the number of images acquired by the imaging device; making the acquired 2D images have sufficient information to support the sparse reconstruction of the target tissue.

4. The method for guiding the positioning of an imaging device based on sparse reconstruction according to claim 3, wherein: The step of using the imaging device to automatically acquire 2D images of a specified area of a simulated patient from several sparse angles specifically includes: The imaging device is a fixed table combined with a movable C-arm imaging device; the simulated patient is placed on the fixed table; the movable C-arm is used to acquire images at preset sparse angles; Alternatively, the imaging device is an imaging device with a rotatable table combined with a fixed radiation source / detector; the simulated patient is placed on the rotatable table; the rotatable table is rotated to acquire images at preset sparse angles.

5. The imaging device positioning guiding method based on sparse reconstruction according to claim 1, characterized in that Adopting a sparse reconstruction algorithm to perform 3D virtual modeling on the selected target tissue; Specifically including: Through a generative adversarial network, candidate volume data of the target tissue can be generated by the generator in iterative reconstruction; Then compare it with the 2D image, extract feature data from the 2D image of the sparse view; decode the feature data to predict the intensity of three-dimensional points; guide the reconstruction to be more realistic based on the intensity predicted by the three-dimensional points.

6. The imaging device positioning guiding method based on sparse reconstruction according to claim 1, wherein The step of registering the 3D modeling image with the 2D image and calculating the first conversion relationship between the visual coordinate system and the motion coordinate system of the imaging device specifically includes: Pre-calibrate the second conversion relationship between the imaging coordinate system and the motion coordinate system; Register the 3D modeling image with the 2D image, calculate the degree-of-freedom transformation matrix generated during the registration process, and obtain the inverse matrix of the degree-of-freedom transformation matrix to acquire the third transformation relationship between the imaging coordinate system and the visual coordinate system; Calculate the first transformation relationship based on the second transformation relationship and the third transformation relationship.

7. The imaging device positioning guiding method based on sparse reconstruction according to claim 1 or 6, characterized in that The registering of the 3D modeling image with the 2D image specifically includes: For the 3D modeling image, calculate its corresponding 2D projection image by means of orthographic projection; Rigidly register the 2D projection image with the 2D image; mark the target tissue on the registered 2D image.

8. The method for guiding the positioning of an imaging device based on sparse reconstruction according to claim 1, wherein, The registering of the 3D modeling image with the 3D standard model of the target tissue to calculate the first transformation relationship between the visual coordinate system and the motion coordinate system of the imaging device specifically includes: Pre-calibrate the 3D standard model and the motion coordinate system; Extract features and perform feature matching on the 3D modeling image and the 3D standard model respectively, and solve the transformation matrix between the two; Calculate the first transformation relationship based on the pre-calibrated result of the 3D standard model and the transformation matrix.

9. An imaging device positioning guidance system based on sparse reconstruction, characterized in that, The system is used to implement the steps of the method according to any one of claims 1-8; it includes: An image acquisition module, configured to use an imaging device to automatically acquire 2D images of a specified area of a simulated patient from several sparse angles; A selection module, configured to select the target tissue to be modeled on the 2D image; A 3D reconstruction module, configured to perform 3D virtual modeling on the selected target tissue by using a sparse reconstruction algorithm to form a 3D modeling image of the target tissue; A registration module, configured to register the 3D modeling image with the 2D image and calculate the first transformation relationship between the visual coordinate system and the motion coordinate system of the imaging device; Or, the registration module is configured to register the 3D modeling image with the 3D standard model of the target tissue and calculate the first transformation relationship between the visual coordinate system and the motion coordinate system of the imaging device; A pose adjustment module, configured to adjust the initial pose of the 3D modeling image, calculate the target pose of the imaging device corresponding to the adjusted pose of the 3D modeling image based on the first transformation relationship and the adjusted pose of the 3D modeling image; control the imaging device to reach the target pose to obtain the expected imaging perspective of the target tissue.

10. An imaging device, characterized in that, The imaging device includes the imaging device pose guiding system based on sparse reconstruction according to claim 9.

Images