A method, apparatus, program product, and medium for position correction of a surgical instrument

Abstract

The application provides a position correction method, device, program product and medium of a surgical instrument, a multi-stage triggering strategy from no-radiation monitoring, low-dose two-dimensional correction to high-dose three-dimensional correction is constructed, first, the surface point cloud obtained by the no-radiation three-dimensional imaging device is used for registration, when the target object only occurs rigid transformation, zero-radiation real-time correction is realized; when non-rigid deformation occurs, only trigger low-dose two-dimensional X-ray imaging once, and through converting the transformation relationship of two-dimensional registration into a three-dimensional transformation relationship, the low-cost update of the navigation matrix is realized; only in the case that the deformation is complex and cannot be fitted by a two-dimensional model, three-dimensional image resampling is enabled as the final guarantee. Through this hierarchical processing architecture, under the premise of ensuring high-precision surgical navigation, the radiation exposure risk of patients and medical staff is maximally reduced, and the technical problem that navigation precision and radiation dose are difficult to balance is effectively solved.

Description

Technical Field

[0001] This application relates to the field of surgical navigation technology, and in particular to a method, device, program product and medium for position correction of surgical instruments. Background Technology

[0002] In image-guided minimally invasive surgery, especially in spinal surgery, preoperative three-dimensional imaging data (such as computed tomography (CT) or cone-beam computed tomography (CBCT) images) provide crucial anatomical information for surgeons to plan surgical pathways and locate lesions. To ensure the accurate implementation of preoperative planning during surgery, precise registration and calibration of the spatial positions of surgical instruments with the preoperative three-dimensional image coordinate system are necessary. Common techniques include implanting invasive markers for optical navigation, ultrasound-based registration navigation, and repeated intraoperative three-dimensional CT scans to update registration information.

[0003] For methods involving the implantation of invasive markers, these markers need to be placed on the patient's skin or bone. This not only introduces additional trauma and infection risks to the patient, but also, during surgery, if the spine deforms due to changes in patient positioning or surgical manipulation, the spatial correspondence between these markers and deep bony structures can change unpredictably, introducing registration errors. For ultrasound-guided methods, due to the low signal-to-noise ratio and limited resolution of ultrasound images, and their susceptibility to operator technique and soft tissue interference, the registration error between ultrasound and CT images is usually large, making it difficult to meet the millimeter-level precision requirements of spinal surgery. While repeated intraoperative 3D CT scans can obtain the most accurate anatomical information at the current moment, each scan exposes patients and medical staff to a high dose of ionizing radiation. Frequent use during surgery significantly increases the risk of radiation exposure, posing potential harm to the patient's health. Therefore, how to achieve high-precision, real-time correction of surgical instrument positions for anatomically deformable structures such as the spine while avoiding or minimizing intraoperative radiation dose is a pressing technical challenge in the field of image-guided surgical navigation. Summary of the Invention

[0004] This application provides a method, device, procedure product, and medium for calibrating the position of surgical instruments, aiming to solve the problems existing in the aforementioned related technologies.

[0005] According to a first aspect of the embodiments of this application, a method for calibrating the position of a surgical instrument is provided, the method comprising: The three-dimensional information of the target object's surface before surgery was obtained using a non-radioactive three-dimensional imaging device, and the first point cloud was generated. The non-radioactive three-dimensional imaging device is used to acquire three-dimensional information of the surface of the target object in real time during surgery, and a second point cloud is generated. Register the first point cloud and the second point cloud to determine the positional difference between the registered second point cloud and the first point cloud. Based on the positional differences, determine whether the target object undergoes only a rigid transformation; If only a rigid transformation occurs, a real-time correction matrix is ​​calculated based on the transformation relationship between the second point cloud and the first point cloud and the pre-calibrated initial registration relationship, and the position of the surgical instrument is corrected using the real-time correction matrix; wherein, the initial registration relationship is the registration relationship between the preoperative three-dimensional image of the target object acquired by the X-ray imaging device and the coordinate system of the surgical instrument. If a non-rigid transformation occurs, the X-ray imaging device is controlled to acquire a two-dimensional image of the current target object. A two-dimensional transformation model containing only translation and scaling is used to register the two-dimensional image with the front projection image of the preoperative three-dimensional image from the same viewpoint, and the similarity between the registered two-dimensional image and the front projection image is determined. Based on the similarity, it is determined whether the deformation of the target object can be fitted by the two-dimensional transformation model; If a fit is possible, the two-dimensional transformation relationship between the two-dimensional image and the front projection image is converted into a three-dimensional transformation relationship. A real-time correction matrix is ​​calculated based on the three-dimensional transformation relationship and the initial registration relationship, and the position of the surgical instrument is corrected using the real-time correction matrix. If a fit cannot be achieved, the X-ray imaging device is controlled to re-acquire a three-dimensional image of the current target object, and the position of the surgical instruments is corrected based on the re-acquired three-dimensional image.

[0006] According to a second aspect of the embodiments of this application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method described in the first aspect.

[0007] According to a third aspect of the embodiments of this application, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0008] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided, on which a computer program is stored, wherein the computer program, when executed by a processor, implements the steps of the method described in the first aspect.

[0009] The technical solution provided in this application can achieve the following beneficial effects: The surgical instrument position correction method provided in this application constructs a multi-level triggering strategy from radiation-free monitoring and low-dose two-dimensional correction to high-dose three-dimensional correction. This method first uses surface point clouds acquired by a radiation-free three-dimensional imaging device for registration, achieving real-time correction with zero radiation when the target object undergoes only rigid transformation; when non-rigid deformation occurs, only one low-dose two-dimensional X-ray imaging is triggered, and the transformation relationship of the two-dimensional registration is converted into a three-dimensional transformation relationship to achieve low-cost updating of the navigation matrix; only when the deformation is extremely complex and cannot be fitted by a two-dimensional model is three-dimensional image resampling used as a final guarantee. Through this hierarchical processing architecture, this application minimizes the radiation exposure risk to patients and medical staff while ensuring high-precision surgical navigation, effectively solving the technical problem of balancing navigation accuracy and radiation dose in related technologies. Attached Figure Description

[0010] Figure 1 This is a schematic diagram of the structure of a surgical navigation system shown in an exemplary embodiment of this application; Figure 2 This is a schematic flowchart illustrating a method for calibrating the position of a surgical instrument, as shown in an exemplary embodiment of this application. Figure 3 This is a schematic diagram illustrating an exemplary embodiment of the present application of a process for generating a point cloud of the spine surface based on an RGBD camera; Figure 4 This is a schematic diagram illustrating a process for extracting surface marker lines of the spine, as shown in an exemplary embodiment of this application. Figure 5 This is a schematic flowchart illustrating another surgical instrument position correction method according to an exemplary embodiment of this application; Figure 6 This is a schematic diagram of the structure of an electronic device shown in an exemplary embodiment of this application. Detailed Implementation

[0011] The core flaw of navigation schemes based on implanted invasive markers lies not merely in the invasiveness of the procedure. A deeper technical root lies in the implicit assumption that this approach is often invalid in spinal surgery: that the spatial relationship between skin or superficial bony markers and deeper surgical targets (such as vertebrae) is rigid and constant. However, the spine itself is a flexible chain structure composed of multiple vertebrae connected by intervertebral discs and ligaments. When a patient changes from a prone position during preoperative CT scans to a position during surgery due to anesthesia, muscle relaxation, or placement on different operating tables, the overall curvature of the spine undergoes significant and non-uniform deformation. This causes unpredictable relative displacement of the implanted markers relative to the deeper vertebrae, fundamentally undermining the "rigid reference" used to construct the registration matrix. For ultrasound navigation methods, the fundamental reason for insufficient accuracy lies in the inherent physical differences between the two imaging modalities: ultrasound imaging relies on differences in acoustic impedance at tissue interfaces, while CT imaging relies on the tissue's attenuation coefficient for X-rays. In the spinal region, the interface between bony structures and soft tissues is complex. Ultrasound waves passing through the cortical bone produce strong reflections, refractions, and acoustic shadowing, resulting in numerous artifacts and signal loss areas in the images, making it difficult to establish a stable and accurate one-to-one correspondence with CT images. While repeated intraoperative 3D CT scans can obtain precise current anatomical structures, the cumulative radiation dose from each scan is significant. In complex surgeries requiring multiple updates to navigation information, the total radiation received by the patient may far exceed safe limits. This is the fundamental reason why this method cannot be used as a routine real-time navigation tool in clinical practice.

[0012] Based on this, and to address the problems existing in the aforementioned related technologies, this application provides a method for calibrating the position of surgical instruments. This method constructs a multi-level triggering strategy, from radiation-free monitoring and low-dose two-dimensional calibration to high-dose three-dimensional calibration. First, it uses surface point clouds acquired by a radiation-free three-dimensional imaging device for registration, achieving real-time calibration with zero radiation when the target object undergoes only rigid transformation. When non-rigid deformation occurs, only one low-dose two-dimensional X-ray imaging is triggered, and the transformation relationship of the two-dimensional registration is converted into a three-dimensional transformation relationship, achieving low-cost updates to the navigation matrix. Only when the deformation is extremely complex and cannot be fitted by a two-dimensional model is three-dimensional image resampling used as a final safeguard, thereby minimizing the radiation exposure risk to patients and medical personnel while ensuring high-precision surgical navigation.

[0013] The technical solution of this application will now be described in detail with reference to specific embodiments and accompanying drawings.

[0014] Figure 1 A schematic diagram of the surgical navigation system used in the surgical instrument position correction method provided in this application embodiment. (See attached diagram.) Figure 1As shown, the system includes: a radiation-free three-dimensional imaging device 100, an X-ray imaging device 200, an optical tracking system 300, surgical instruments 400, and electronic equipment 500.

[0015] A radiation-free 3D imaging device 100 is used to acquire 3D information of the surface of a target object (e.g., the skin on the back of a patient's spine) before and during surgery to generate corresponding point cloud data. This device can be an RGBD camera capable of simultaneously acquiring color and depth images; in other embodiments, it can also employ a binocular stereo vision camera, LiDAR, or a structured light-based 3D scanner. The radiation-free 3D imaging device 100 is typically mounted above the surgical area so that its field of view covers the surface area of ​​the target object.

[0016] X-ray imaging device 200 is used to acquire X-ray images of the target object to provide preoperative three-dimensional reference images and intraoperative two-dimensional or three-dimensional correction images. This device can be a mobile C-arm, O-arm, or conventional computed tomography (CT) scanner. Preoperatively, X-ray imaging device 200 performs a complete scan around the target object to reconstruct high-precision three-dimensional CT or CBCT images. Intraoperatively, when radiation-free three-dimensional imaging device 100 detects non-rigid deformation of the target object, electronic device 500 controls X-ray imaging device 200 to perform different levels of imaging based on the complexity of the deformation: if the deformation is relatively simple and can be fitted by a two-dimensional translation / scaling model, X-ray imaging device 200 is controlled to quickly take a two-dimensional film (e.g., anteroposterior or lateral view) to obtain low-dose projection information at the current moment; if the deformation is extremely complex and cannot be fitted by a two-dimensional model, X-ray imaging device 200 is controlled to perform a complete multi-angle scan to re-acquire and reconstruct the three-dimensional image in the current state as the final navigation reference.

[0017] An optical tracking system 300 is used to monitor the position and orientation of a surgical instrument 400 in space in real time. This system typically includes one or more optical tracking cameras (e.g., infrared stereo cameras) and tracking markers (e.g., reflective spheres or active light-emitting diodes) rigidly connected to the surgical instrument 400. The optical tracking system 300 is capable of capturing the three-dimensional coordinates of the markers with sub-millimeter precision and transmitting them to an electronic device 500 in real time.

[0018] Surgical instrument 400 is a tool used to perform operations according to surgical needs. For example, in minimally invasive spinal surgery, the instrument may be a puncture needle, guide needle, pedicle screwdriver, bone drill, or implant delivery device. In order to be recognized by the optical tracking system 300, the surgical instrument 400 is fixed with a detachable or non-detachable tracking marker. The geometric relationship between the marker and the instrument tip is pre-calibrated, so that the spatial coordinates of the instrument tip can be calculated based on the pose of the marker.

[0019] The electronic device 500 is communicatively connected to the radiation-free three-dimensional imaging device 100, the X-ray imaging device 200, the optical tracking system 300, and the surgical instrument 400. The electronic device 500 can be a dedicated medical imaging workstation, a high-performance computer, an embedded controller, or a cloud server, etc. Internally, it includes at least one processor and a memory. The memory stores a computer program, which, when executed by the processor, implements the various steps of the surgical instrument position correction method described in this application. The electronic device 500 can also be connected to a display (…). Figure 1 (Not shown), used to display preoperative 3D images, the superimposed positions of intraoperative instruments, and navigation guidance information in real time.

[0020] Before the surgery begins, the electronic device 500 uses the X-ray imaging device 200 to acquire a three-dimensional image of the target object, and uses a reference plate or probe to calibrate the initial registration relationship between the coordinate system of this three-dimensional image and the coordinate system of the optical tracking system 300 (i.e., the surgical instruments). Simultaneously, the non-radioactive three-dimensional imaging device 100 acquires the initial three-dimensional information of the target object's surface, generating a first point cloud. During the surgery, the electronic device 500 acquires the second point cloud acquired by the non-radioactive three-dimensional imaging device 100 in real time and registers it with the first point cloud. Based on the registration result, the deformation state of the target object is determined (rigid transformation only or non-rigid deformation). When it is determined to be a rigid transformation only, the electronic device 500 updates the navigation matrix based on the transformation relationship obtained from the point cloud registration, without needing to activate the X-ray imaging device 200 throughout the entire process. When a non-rigid deformation is detected, the electronic device 500 uses a multi-level triggering strategy to control the X-ray imaging device 200 to selectively acquire two-dimensional flat films or complete three-dimensional images according to the complexity of the deformation, and calculates a new correction matrix accordingly. Finally, the real-time position of the surgical instrument 400 is accurately mapped onto the preoperative or updated three-dimensional image, providing doctors with precise and low-radiation navigation support.

[0021] Figure 2 This is a schematic flowchart illustrating a method for calibrating the position of a surgical instrument according to an exemplary embodiment of this application. Figure 2 As shown, the surgical instrument position correction method provided in this application may include the following steps S201 to S209.

[0022] Step S201: Use a non-radioactive 3D imaging device to acquire the 3D information of the target object surface before surgery and generate the first point cloud.

[0023] During the preoperative preparation phase, electronic devices can control a radiation-free 3D imaging device to scan the surface of the target object to obtain its 3D spatial information and generate a first point cloud based on this information. This first point cloud is a data set consisting of a large number of points with 3D spatial coordinates (e.g., X, Y, Z coordinates). Its purpose is to provide a radiation-free and safe reference for the surface morphology of the target object, which is used for comparison with the data acquired in real time during the operation.

[0024] In this embodiment, a radiation-free 3D imaging device refers to any device capable of acquiring 3D spatial information of an object's surface without exposing the patient to ionizing radiation. The target object can be an anatomical structure prone to deformation during surgery, such as the spine, liver, or lungs. Specifically, the radiation-free 3D imaging device can be an RGBD camera. An RGBD camera is a composite sensor capable of simultaneously acquiring color images and depth information of a target scene. Its imaging principle can be based on structured light, i.e., calculating depth by projecting a specific coded light spot pattern and analyzing its deformation on the object's surface; or it can be based on the time-of-flight method, i.e., calculating depth by measuring the time difference between the emitted infrared light pulse and the received reflected light pulse. Besides an RGBD camera, this device can also be replaced with other radiation-free devices capable of generating point clouds, such as a binocular stereo vision camera (acquiring depth information by calculating parallax using two or more ordinary color cameras), a lidar (constructing a 3D point cloud by emitting a laser beam and measuring the reflection time), or a dedicated structured light-based scanner. After the first point cloud is generated, it will be stored in the electronic device's memory or non-volatile memory for subsequent registration steps.

[0025] Step S202: Use the non-radioactive three-dimensional imaging device to acquire the three-dimensional information of the surface of the target object in real time during the operation, and generate a second point cloud.

[0026] During the surgery, the electronic device can continuously or on-demand utilize the same radiation-free three-dimensional imaging device as in step S201, and scan the surface of the target object in real time using the same data acquisition method as generating the first point cloud, to generate a second point cloud. This second point cloud reflects the actual surface morphology of the target object at the current moment, after being affected by surgical positioning adjustments and surgical operations. For example, when the target object is the spine, the patient's prone position on the operating table may differ from the position during the preoperative CT scan. Simultaneously, the pressing and spreading operations of surgical instruments may also cause changes in the morphology of the skin surface of the spine; all these changes will be captured by the second point cloud. The core function of this step is that it provides a radiation-free dynamic information source for subsequent real-time comparison with the preoperative baseline (i.e., the first point cloud), thereby achieving continuous monitoring of the intraoperative surface morphology of the target object without introducing any ionizing radiation.

[0027] Step S203: Register the first point cloud and the second point cloud to determine the positional difference between the registered second point cloud and the first point cloud.

[0028] To quantify the spatial changes of the target object from preoperative to intraoperative stages, the electronic device needs to register a first point cloud with a second point cloud. The goal of point cloud registration is to find an optimal spatial transformation relationship that maximizes the spatial overlap between the transformed second and first point clouds. In this embodiment, the transformation relationship can be a rigid body transformation matrix, including rotation and translation components. The registration algorithm can employ the iterative nearest-point algorithm, whose basic principle is: in one iteration, for each point in the second point cloud, find its nearest corresponding point in the first point cloud, then calculate a rigid body transformation that minimizes the sum of squared distances between these two sets of corresponding points, and apply this transformation to update the position of the second point cloud. This process is repeated iteratively until convergence. Besides the ICP algorithm, other point cloud registration algorithms can also be used, such as the normal distribution transformation algorithm. This algorithm divides the point cloud space into several voxel grids, calculates the probability density function of the point cloud within each grid, and then solves for the transformation matrix by optimizing the similarity between the probability density functions of the two point clouds. In addition, for objects with obvious features, coarse registration can be performed first using local feature descriptors such as fast point feature histograms, providing a good initial position for fine registration algorithms such as ICP.

[0029] After registration, the electronic device can calculate the positional difference between the transformed second point cloud and the first point cloud. This positional difference can be quantified using various metrics, such as calculating the root mean square error of the Euclidean distance between all corresponding points, the average of all distances, the proportion of points exceeding a preset distance threshold, or the average Hausdorff distance between point clouds, etc. The calculated value is the positional difference value, which intuitively reflects the shape changes remaining on the surface of the target object after eliminating overall rigid motion. It is a key quantitative basis for subsequent judgment of whether the target object has undergone deformation.

[0030] Step S204: Determine whether the target object only undergoes a rigid transformation based on the positional difference.

[0031] In actual surgery, spatial changes in the target object can be categorized into two fundamentally different types: rigid transformations, which involve only overall translation and rotation while maintaining the same shape, and non-rigid transformations (i.e., deformation), which involve changes in shape such as stretching, bending, or torsion. Since the positional difference calculated in step S203 is the residual error after eliminating the optimal rigid body registration between the two point clouds, its magnitude directly reflects whether there are shape changes on the target object's surface that cannot be explained by rigid body transformations. If the positional difference is small, it indicates that the two point clouds have highly overlapped after rigid body registration, and the target object has only undergone overall rigid displacement; conversely, if the positional difference is large, it indicates significant deformation. Based on this principle, the electronic device can classify the physical state of the target object according to the positional difference, thereby determining which correction strategy to use subsequently. For example, if the entire spine only undergoes translation and rotation relative to the non-radioactive 3D imaging device due to the overall movement of the patient's body position, but its physiological curvature remains unchanged, it is determined that only a rigid transformation has occurred. Conversely, if the curvature of the spine changes, such as an increase or decrease in lumbar lordosis, it is determined that a non-rigid transformation has occurred.

[0032] In one specific embodiment, this judgment process is implemented through a preset threshold. Specifically, if the positional difference is less than or equal to a first preset threshold, it is determined that the target object has only undergone a rigid transformation; if the positional difference is greater than the first preset threshold, it is determined that the target object has undergone a non-rigid transformation. The first preset threshold can be an empirical value, the magnitude of which is related to the physiological characteristics of the target object, the point cloud accuracy, and the precision requirements of the surgery. For example, for thoracic spine surgery, the first preset threshold can be set to 2 mm: if the calculated root mean square error of the positional difference is less than or equal to 2 mm, it is considered that the overall displacement of the spine has been compensated through rigid body registration, and the remaining error is within an acceptable range, i.e., only a rigid transformation has occurred; if the root mean square error is greater than 2 mm, it means that there is a residual error that cannot be explained by rigid body transformation, indicating that the spine may have deformed. This first preset threshold can be determined by statistical analysis of a representative set of samples before surgery, or it can be set by the doctor based on the specific type of surgery and their own experience. Besides using absolute distance thresholds, dynamic thresholding can also be employed. For example, the convergence trend of the first and second point clouds during ICP registration can be correlated. When the iterative error fails to decrease to a stable baseline value after multiple iterations, this baseline value is multiplied by a coefficient to serve as the first preset threshold. Another approach utilizes the distribution characteristics of the residuals after point cloud registration: if the residuals exhibit a random Gaussian distribution, it is considered a rigid transformation; if the residuals exhibit a non-random, structured pattern in space, it is considered a non-rigid transformation. This classification determines the branching direction of subsequent correction processes.

[0033] Step S205: If only rigid transformation occurs, calculate the real-time correction matrix based on the transformation relationship between the second point cloud and the first point cloud and the pre-calibrated initial registration relationship, and use the real-time correction matrix to correct the position of the surgical instrument.

[0034] When the electronic device determines that the target object has only undergone a rigid transformation, it means that its internal relative structure has not changed. Therefore, only a holistic rigid update of the navigation coordinate system established before surgery is needed. During the registration process in step S203, the electronic device has calculated the rigid transformation relationship of the second point cloud relative to the first point cloud, denoted as T1. At the same time, the electronic device acquires the pre-calibrated initial registration relationship T0. T0 is the registration relationship between the preoperative three-dimensional image of the target object acquired by the X-ray imaging device (e.g., C-arm, O-arm, or conventional CT scanner) and the coordinate system of the surgical instruments. The calibration of T0 can be completed during the preoperative preparation stage. For example, a reference plate with multiple opaque X-ray marker spheres can be used, which can be simultaneously identified by the radiation-free three-dimensional imaging device and the X-ray imaging device, thereby establishing a mapping between the coordinate systems of the two devices; or, a handheld probe can be used to contact specific points in the image and their corresponding points in reality to perform point-to-point registration. Based on the superposition principle of rigid transformation, the electronic device can calculate a new real-time correction matrix T for the current surgical procedure, with the formula T = T0 × T1.

[0035] When using this real-time correction matrix to correct the position of surgical instruments, the electronic device first acquires the current position coordinates P_tool of the surgical instrument under its tracking system (e.g., an optical tracking system). Then, it calculates the corresponding coordinates of the surgical instrument in the preoperative three-dimensional image coordinate system according to the formula P_target=T×P_tool, thereby displaying the relative positional relationship between the tip of the surgical instrument and the lesion in real time on the navigation display. The surgeon adjusts the operating path of the surgical instrument based on this displayed information to ensure it accurately reaches the target position. Since the entire correction process only uses information acquired by a radiation-free three-dimensional imaging device and does not introduce any additional ionizing radiation, this step achieves high-precision navigation under zero-radiation conditions.

[0036] Step S206: If a non-rigid transformation occurs, the X-ray imaging device is controlled to acquire a two-dimensional image of the current target object. A two-dimensional transformation model containing only translation and scaling is used to register the two-dimensional image with the preoperative three-dimensional image from the same viewpoint and determine the similarity between the registered two-dimensional image and the preoperative projection image.

[0037] When electronic equipment determines that a target object has undergone a non-rigid transformation (i.e., deformation), the rigid transformation of the surface point cloud alone is insufficient to accurately describe the changes in its internal structure. This is because the surface point cloud only reflects the surface morphology, while the deformation of the target object (such as the spine) mainly manifests as the relative displacement and curvature changes between internal structures (such as vertebrae). These internal changes cannot be fully captured by surface optical signals. In this case, X-ray information is needed to directly observe the internal structure. However, performing a complete 3D CT scan immediately after deformation occurs would introduce a high radiation dose. Considering the varying complexity of deformations, some deformations may be relatively simple and can be described and compensated for through 2D projection registration. Therefore, electronic equipment can first control the X-ray imaging device to acquire a low-dose 2D image, with a radiation dose far lower than that of a complete 3D CT scan, typically only a fraction of the latter. By registering this 2D image with the anterior projection of the preoperative 3D image, it is possible to preliminarily determine whether the deformation can be fitted by a simple 2D transformation model, thus avoiding a higher radiation dose 3D scan if a fit is possible.

[0038] Specifically, the electronic device triggers and controls the X-ray imaging equipment (e.g., a C-arm) to capture a two-dimensional image at a specific angle (e.g., 0°, i.e., orthogonal view). Simultaneously, the electronic device uses techniques such as digital reconstruction of radiographic images to calculate and generate a pre-projection image from the pre-operative three-dimensional image, under the same spatial geometric parameters (i.e., the positions of the X-ray tube and detector). Subsequently, a two-dimensional transformation model containing only translation (dx, dy) and scaling (s) is used to register the acquired two-dimensional image, aligning it as closely as possible with the pre-projection image. The reason for using only a translation and scaling transformation model at this stage is that the initial intention of this step is to attempt to fit the current deformation with a model that has the fewest parameters and the strongest robustness, assuming that the deformation has not exceeded the model's expressive power. If this simple model achieves good registration results (i.e., high similarity), it indicates that the current deformation is indeed relatively simple and can be approximated by translation and scaling, thus avoiding the use of more complex models or higher-dose scans; if the similarity is low, it indicates that the deformation exceeds the model's fitting capability, requiring subsequent higher-level corrections. The algorithm used to perform this registration can be a registration method based on grayscale information, such as gradient descent combined with mutual information as a similarity measure, or a feature-based method, such as extracting the contours or corners of the cone for matching.

[0039] Step S207: Determine whether the deformation of the target object can be fitted by the two-dimensional transformation model based on the similarity.

[0040] After registration, the electronic device calculates the similarity between the registered 2D flat image and the pre-projected image. This similarity is used to assess whether the 2D transformation model used is sufficient to describe the current deformation, that is, to determine whether the current deformation can be approximated by the translation and scaling model. The underlying logic is as follows: if the 2D flat image and the pre-projected image are still very similar after optimal translation and scaling transformation, it means that the difference between the two has been largely explained by the transformation model, and the residual dissimilarity is low. Therefore, the current deformation can be considered to be fit by the model. Conversely, if there are still significant structural differences between the two after optimal registration, it indicates that the deformation contains complex components that cannot be described by translation and scaling (such as non-uniform distortion, rotation, etc.), and a higher-level correction strategy is required.

[0041] In one specific embodiment, the similarity can be characterized using mutual information. Mutual information is a metric in information theory used to measure the strength of statistical dependence between two images; the higher the value, the more similar the two images are. Specifically, if the mutual information is greater than or equal to a second preset threshold, it is determined that the deformation of the target object can be fitted by the two-dimensional transformation model; if the mutual information is less than the second preset threshold, it is determined that it cannot be fitted. The second preset threshold can be optimized and determined by offline testing on a large number of spine samples with different degrees of deformation; for example, it can be set to 0.75 (normalized mutual information value).

[0042] In another embodiment, the similarity can be characterized using grayscale differences, such as root mean square error or mean absolute difference. Grayscale differences offer the advantage of being simple and fast to calculate, making them suitable for scenarios with extremely high real-time requirements. However, they typically require histogram matching preprocessing of the two images to eliminate the influence of overall brightness differences. The smaller the grayscale difference, the more similar the two images are. In this case, if the grayscale difference is less than or equal to a third preset threshold, it is determined that the image can be fitted; if the grayscale difference is greater than the third preset threshold, it is determined that the image cannot be fitted. For example, the third preset threshold can be set to 50 grayscale levels (for an 8-bit image).

[0043] In addition to the two measurement methods mentioned above, similarity can also be measured by other methods such as correlation coefficient, structured similarity index (SSIM), or feature similarity based on local phase consistency. These methods can be used alone or in combination to improve the robustness of the judgment.

[0044] Step S208: If a fit is possible, the two-dimensional transformation relationship between the two-dimensional image and the front projection image is converted into a three-dimensional transformation relationship. A real-time correction matrix is ​​calculated based on the three-dimensional transformation relationship and the initial registration relationship, and the position of the surgical instrument is corrected using the real-time correction matrix.

[0045] When the electronic device determines that a two-dimensional transformation model is sufficient to fit the current deformation, it indicates that although the target object (e.g., the spine) has deformed, this deformation manifests in the two-dimensional projection space of X-rays as a relatively simple change that can be described by a translation and scaling model. In this case, this two-dimensional transformation relationship can be converted into a three-dimensional transformation relationship, thus correcting the entire three-dimensional navigation coordinate system using only a single two-dimensional image (low radiation), avoiding the high-dose radiation associated with rescanning a three-dimensional CT scan.

[0046] In a specific embodiment, the transformation process is performed according to the following rules. Since it has been determined in step S207 that the current deformation can be fitted by a two-dimensional transformation model that only includes translation and scaling, this means that in the orthogonal projection direction, the deformation is mainly manifested as the shift in overall position and the change in magnification caused by displacement along the depth direction, while the rotation around each axis and the volume change of the object itself are not significant. Based on this determination, firstly, the rotation component of the three-dimensional transformation relationship is set as an identity matrix, assuming that the deformation does not include significant rotation around the axis; the scaling component in the three-dimensional transformation relationship is also set as an identity matrix, assuming that the size of the three-dimensional object itself has not changed. Secondly, the translation component in the horizontal direction (image X-axis direction) of the three-dimensional transformation relationship is set as the horizontal translation component dx in the two-dimensional transformation relationship, and the translation component in the vertical direction (image Y-axis direction) of the three-dimensional transformation relationship is set as the vertical translation component dy in the two-dimensional transformation relationship. Finally, the translation component in the depth direction (Z-axis, i.e., the direction of X-ray propagation) of the three-dimensional transformation relationship is set as the product of the scaling component s in the two-dimensional transformation relationship and the distance SID from the X-ray source to the detector of the X-ray imaging device, i.e., ΔZ = s × SID. The geometric principle of this transformation is based on similar triangles: when an object moves ΔZ in the depth direction, its projection magnification on the detector changes by SID / (SID-ΔZ). For small deformations, this magnification change is approximately 1 + ΔZ / SID, and its linear part is related to the two-dimensional scaling factor s. Therefore, the approximate translation in the depth direction can be calculated backward from s. After the above transformation, the three-dimensional deformation / displacement matrix T2 is obtained. Then, combined with the initial registration relationship T0, the electronic device calculates a new real-time correction matrix T = T0 × T2. Subsequently, this matrix is ​​applied to the position correction of surgical instruments, in the same way as step S205. In this way, a single low-dose two-dimensional X-ray exposure can achieve rapid updating of the three-dimensional navigation matrix, effectively compensating for navigation errors caused by spinal deformation while significantly reducing radiation dose.

[0047] Step S209: If a fit cannot be achieved, control the X-ray imaging device to re-acquire a three-dimensional image of the current target object, and correct the position of the surgical instruments based on the re-acquired three-dimensional image.

[0048] When the similarity calculated in step S207 is lower than a preset threshold (e.g., mutual information is less than a second preset threshold, or grayscale difference is greater than a third preset threshold), it indicates that the deformation of the target object is very complex, such as relative slippage, rotation, or lateral bending of the vertebrae, which cannot be fitted by a simple two-dimensional translation / scaling model. In this case, if continued attempts are made to correct based on the two-dimensional image, it may lead to serious navigation errors, thereby threatening surgical safety. Therefore, as a safety fallback mechanism, the electronic device can prompt the doctor or automatically trigger a higher-level correction process. Specifically, the electronic device controls the X-ray imaging equipment to perform a complete multi-angle scan (e.g., a standard three-dimensional CBCT scan) to acquire enough data to regenerate a high-precision three-dimensional image in the current state. Subsequently, based on this new three-dimensional image, the electronic device performs an initial and complete registration process, that is, re-establishes the registration relationship T0' between the new three-dimensional image and the surgical instrument coordinate system, and uses T0' as the reference for subsequent navigation. Although this step introduces a relatively high radiation dose, it occurs under extreme circumstances where the deformation exceeds the processing capabilities of the two-dimensional model. As the final safeguard of this scheme, it ensures that the surgery can still obtain reliable high-precision navigation support when complex deformations occur.

[0049] Through steps S201 to S209 above, the surgical instrument position correction method provided in this application constructs a multi-level triggering strategy from radiation-free monitoring and low-dose two-dimensional correction to high-dose three-dimensional correction. This method first utilizes radiation-free surface point cloud registration to achieve zero-radiation correction for most rigid displacements; when non-rigid deformation occurs, only one low-dose two-dimensional X-ray exposure is triggered, and a low-cost update of the navigation matrix is ​​achieved through an innovative two-dimensional to three-dimensional transformation relationship; only in cases of extremely complex deformation is a high-dose three-dimensional scan used as a final safeguard. This hierarchical processing architecture, while ensuring navigation accuracy, minimizes the radiation exposure risk to patients and medical personnel, solving the technical challenge of balancing radiation dose and navigation accuracy in existing technologies.

[0050] Based on the above embodiments, this application has been able to achieve positional correction of surgical instruments for a target object. However, when the target object is specifically the spine, due to its indistinct surface landmarks and susceptibility to soft tissue interference, general point cloud generation methods struggle to stably and accurately extract surface features corresponding to deep bones. Therefore, this application further provides a preferred embodiment suitable for the specific target object of the spine. In this embodiment, the radiation-free three-dimensional imaging device specifically uses an RGBD camera, and the target object is the spine. Accordingly, the generation method of the second point cloud during surgery is exactly the same as that of the first point cloud, i.e., the RGB and depth images of the patient's back are acquired using an RGBD camera during surgery, and the second point cloud is obtained by processing it according to the same steps, which will not be repeated here. The following focuses on describing the generation process of the first point cloud.

[0051] Figure 3 This is a schematic diagram illustrating the process of generating a point cloud of the spine surface based on an RGBD camera according to an exemplary embodiment of this application. Figure 3 As shown, the method includes steps S301 to S304.

[0052] Step S301: Use the RGBD camera to acquire RGB and depth images of the patient's back before surgery.

[0053] In this step, the patient lies prone on the operating table, and an RGBD camera is positioned vertically against their back, simultaneously acquiring high-resolution color images and corresponding depth images. The RGB image provides texture and color information, while the depth image provides the distance information of each pixel from the camera. These two images are pixel-level aligned, meaning that each pixel in the RGB image describes the same spatial location as its corresponding pixel in the depth image.

[0054] Step S302: Use a semantic segmentation network to perform left-right symmetrical segmentation on the RGB image to obtain the left back region and the right back region.

[0055] The purpose of this step is to accurately divide the back image into left and right halves so that the midline of the spine can be located by calculating the midpoint of the left and right boundaries. Since the spine is located in the middle of the back, and the back muscles on both sides are symmetrically distributed in a normal posture, the boundary line between the left and right back regions corresponds to the surface projection line of the spinous processes of the spine. Semantic segmentation networks are deep learning models capable of classifying each pixel in an image. For this application scenario, a typical network is U-Net, whose encoder-decoder structure combined with skip connections can extract deep semantic features without sacrificing spatial resolution, thus accurately segmenting the left and right back regions. It maintains stable segmentation performance even under interference from surgical drapes, drainage tubes, or disinfectant residue. In this application, U-Net is used to distinguish between three categories in the back image: "left back skin," "right back skin," and "background / drape." Training this network requires a large amount of back image data with fine pixel-level annotations, where the boundary lines between the left and right sides of the back (i.e., the position of the midline of the spine) must be carefully drawn. The input to this network is an RGB image, and the output is a classification probability map of the same size as the input. In addition to U-Net, other segmentation networks can also be used, such as DeepLab V3+ (which uses dilated convolutions to expand the receptive field and better capture the overall contour of the back), fully convolutional networks (FCN), or Transformer-based SegFormer (which models global contextual relationships through a self-attention mechanism and is more robust to changes in illumination).

[0056] Step S303: Calculate the midpoint of the boundary of the left back region and the right back region on each row of the image or on each vertical slice along the spine, and perform curve fitting on all midpoints to obtain the spine surface marking line.

[0057] Since the spine is not a perfect straight line but has physiological curvature, directly calculating the midpoints line by line can trace the spine's direction. The specific operation is as follows: In the horizontal direction of the image, scan from left to right to find the first pixel belonging to the right back region, which serves as the right boundary of the left back region; scan from right to left to find the first pixel belonging to the left back region, which serves as the left boundary of the right back region; then calculate the center point of these two boundary points. Repeating this operation yields a series of midpoints distributed along the spine's direction. Because the segmentation result may contain noise or local unevenness, directly connecting these midpoints will result in a jagged, broken line. Therefore, curve fitting is needed for all midpoints, for example, using B-spline curves or Bezier curves for least-squares fitting, to obtain a smooth, continuous curve that accurately reflects the projection position of the spinous processes on the body surface.

[0058] Step S304: Fuse the spinal surface marking lines with the depth image to generate the first point cloud.

[0059] Specifically, the depth value d (the Z-coordinate in the camera coordinate system) of each marked pixel can be obtained from the aligned depth image based on its two-dimensional coordinates (u, v). Then, by combining the intrinsic parameters of the RGBD camera (including focal length, principal point coordinates, etc.), the two-dimensional pixel coordinates (u, v) and depth value d are back-projected into the three-dimensional camera coordinate system to obtain the three-dimensional spatial point coordinates (X, Y, Z). After performing this operation on all points on the marked lines, a series of three-dimensional spatial points are generated, and the set of these points constitutes the first point cloud. This point cloud not only contains the positional information of the spine but also implicitly contains the concave morphology of its back surface (because the depth value reflects the undulation of the spinal groove), providing reliable anatomical features for subsequent point cloud registration. The entire process does not require the implantation of any physical markers into the patient, achieving completely non-invasive spinal localization.

[0060] The above approach cleverly utilizes the prior knowledge of the physiological anatomy that the spine is located in the midline depression of the back, transforming the challenging task of "segmenting the spine" into the simpler and more robust task of "segmenting the left and right sides of the back." By calculating the midpoints of the boundaries of the left and right segmented regions and obtaining smooth marker lines through curve fitting, this method can extract stable features highly correlated with deep bones from the patient's body surface without invasiveness or markers, thus distinguishing it from the conventional approach of directly registering point clouds across the entire back. The latter relies on feature points (such as pores and textures) that are not directly related to bones and are prone to failure when soft tissue deforms. The feature extraction method provided in this embodiment offers a stable and reliable anatomical benchmark for subsequent rigidity transformation assessment, further improving the accuracy and robustness of spinal surgery navigation. Figure 4 The process of extracting the surface marker lines of the spine described above is illustrated by example.

[0061] Furthermore, considering the complex lighting environment in the actual spinal surgery setting, multiple light sources such as shadowless lamps, monitor lights, and endoscopic light sources exist simultaneously, each with varying color temperatures and intensities. This can cause non-uniform lighting interference in the back images captured by the RGB camera, resulting in uneven brightness or glare on the originally uniformly colored back skin. These interferences directly affect the performance of the semantic segmentation network, causing fluctuations in the segmentation boundaries of the left and right back regions, thereby reducing the accuracy of the spinal surface marker lines. Therefore, in some embodiments, before performing symmetrical left-right segmentation of the RGB image using the semantic segmentation network, illumination normalization preprocessing can be performed on the RGB image to eliminate or suppress the effects of illumination variations. This preprocessing can be applied to every frame of the RGB image before and during surgery, thereby improving the stability of segmentation and the quality of subsequent point cloud generation.

[0062] Specifically, illumination normalization can be achieved in various ways. A common method is histogram equalization, which improves the overall brightness distribution by adjusting the image contrast. A more advanced method is based on Retinex theory, which decomposes an image into illuminance and reflectance components. By removing or compressing the illuminance component, the reflectance component, representing the inherent properties of objects, is recovered, such as single-scale or multi-scale Retinex algorithms. Another method is to use adaptive gamma correction, which dynamically adjusts the correction coefficients based on the brightness characteristics of local image regions. Furthermore, in addition to illumination normalization during preprocessing, data augmentation techniques can be used during the training of semantic segmentation networks. This involves artificially adding images simulating various illumination changes to the training samples, such as randomly adjusting brightness, contrast, gamma values, or adding Gaussian noise. This makes the trained model robust to illumination changes, thereby reducing dependence on specific preprocessing steps. These methods can be used individually or in combination, depending on the specific application scenario, to achieve optimal image quality.

[0063] In summary, to provide a clearer understanding of the technical solutions of this application, a specific application example will be used to illustrate the application as a whole. This example is a spinal surgery navigation system equipped with an RGBD camera (as a radiation-free three-dimensional imaging device) and a movable C-arm (as an X-ray imaging device).

[0064] Figure 5 This is a schematic flowchart illustrating another method for position correction of surgical instruments according to an exemplary embodiment of this application. Figure 5 As shown, this embodiment includes the following steps.

[0065] Step S501: Before surgery, RGB and depth images of the patient's back are acquired using an RGBD camera. After illumination normalization preprocessing, the RGB images are input into a semantic segmentation network to obtain segmentation masks for the left and right back regions. The midpoint of the boundary of each row is calculated and fitted to obtain the surface marker line of the spine. The marker line is then fused with the depth image to generate the first point cloud A. At the same time, preoperative three-dimensional CBCT images are acquired using a C-arm and the initial registration matrix T0 is calibrated.

[0066] In step S502, during the operation, the current back RGB image and depth image captured by the RGBD camera are acquired in real time, and the second point cloud B is generated in the same way as in step S501.

[0067] Step S503: Perform point cloud registration between the second point cloud B and the first point cloud A to obtain the rigid transformation matrix T1, and calculate the positional difference after registration.

[0068] Step S504: Compare the position difference with a first preset threshold: If the position difference is less than or equal to the first preset threshold, it is determined that only rigid transformation has occurred, and step S505 is executed; If the position difference is greater than the first preset threshold, it is determined that non-rigid deformation has occurred, and step S506 is executed.

[0069] In step S505, it is determined that only rigid transformation has occurred, the real-time correction matrix T=T0×T1 is calculated, and the position of the surgical instrument is corrected using this matrix. Then, the process returns to step S502 to continue monitoring.

[0070] Step S506: Determine that a non-rigid deformation has occurred, control the C-arm to acquire a two-dimensional image, and generate a front projection image from the same viewpoint from the preoperative three-dimensional CBCT image; use a two-dimensional transformation model that only includes translation and scaling, and use mutual information as a similarity measure to register the two-dimensional flat film with the front projection image, and determine the mutual information value after registration.

[0071] Step S507: Compare the mutual information value with the second preset threshold: If the mutual information value is greater than or equal to the second preset threshold, it is determined that the deformation can be fitted by the two-dimensional model, and step S508 is executed; If the mutual information value is less than the second preset threshold, it is determined that the deformation cannot be fitted, and step S509 is executed.

[0072] In step S508, it is determined that the deformation can be fitted by a two-dimensional model. The two-dimensional transformation relationship is converted into a three-dimensional transformation relationship to obtain a three-dimensional transformation matrix T3. The real-time correction matrix T=T0×T3 is calculated and the position of the surgical instrument is corrected using this matrix. Then, the process returns to step S502 to continue monitoring.

[0073] In step S509, if it is determined that the deformation is too complex to be fitted by a two-dimensional model, the C-arm is controlled to reacquire a three-dimensional image of the current target object, and the initial registration matrix is ​​recalibrated based on the new three-dimensional image. The position of the surgical instruments is corrected using the recalibrated registration matrix, and then the process returns to step S502 to continue monitoring.

[0074] Through the specific application examples described above, the surgical instrument position correction method provided in this application can adaptively select the optimal correction strategy under different degrees of deformation: from radiation-free rigid correction, to deformation correction requiring only a single low-dose two-dimensional X-ray exposure, and finally to the ultimate guarantee of high-dose three-dimensional scanning. This method minimizes the use of intraoperative ionizing radiation while ensuring navigation accuracy, providing a safe, precise, and efficient navigation solution for surgeries on easily deformable sites such as the spine.

[0075] The scanned image reconstruction method provided in this application can be implemented by an electronic device executing a corresponding computer program. Specifically, the electronic device loads the computer program into non-volatile memory, and the processor reads these computer program instructions into memory for execution. Figure 6 As shown, the hardware structure of this electronic device may include a processor 601, a network interface 602, memory 603, and non-volatile memory 604. In addition, the electronic device may include other hardware components according to actual functional requirements, which will not be elaborated here.

[0076] Furthermore, the scanned image reconstruction method provided in this application can also be implemented by a processor executing a computer program contained in a computer program product, or by a processor executing a computer program stored on a computer-readable storage medium.

Claims

1. A method for calibrating the position of a surgical instrument, characterized in that, The method includes: The three-dimensional information of the target object's surface before surgery was obtained using a non-radioactive three-dimensional imaging device, and the first point cloud was generated. The non-radioactive three-dimensional imaging device is used to acquire three-dimensional information of the surface of the target object in real time during surgery, and a second point cloud is generated. Register the first point cloud and the second point cloud to determine the positional difference between the registered second point cloud and the first point cloud. Based on the positional differences, determine whether the target object undergoes only a rigid transformation; If only a rigid transformation occurs, a real-time correction matrix is ​​calculated based on the transformation relationship between the second point cloud and the first point cloud and the pre-calibrated initial registration relationship, and the position of the surgical instrument is corrected using the real-time correction matrix; wherein, the initial registration relationship is the registration relationship between the preoperative three-dimensional image of the target object acquired by the X-ray imaging device and the coordinate system of the surgical instrument. If a non-rigid transformation occurs, the X-ray imaging device is controlled to acquire a two-dimensional image of the current target object. A two-dimensional transformation model containing only translation and scaling is used to register the two-dimensional image with the front projection image of the preoperative three-dimensional image from the same viewpoint, and the similarity between the registered two-dimensional image and the front projection image is determined. Based on the similarity, it is determined whether the deformation of the target object can be fitted by the two-dimensional transformation model; If a fit is possible, the two-dimensional transformation relationship between the two-dimensional image and the front projection image is converted into a three-dimensional transformation relationship. A real-time correction matrix is ​​calculated based on the three-dimensional transformation relationship and the initial registration relationship, and the position of the surgical instrument is corrected using the real-time correction matrix. If a fit cannot be achieved, the X-ray imaging device is controlled to re-acquire a three-dimensional image of the current target object, and the position of the surgical instruments is corrected based on the re-acquired three-dimensional image.

2. The method according to claim 1, characterized in that, Determining whether the target object undergoes only a rigid transformation based on the positional differences specifically includes: If the positional difference is less than or equal to a first preset threshold, the target object undergoes only a rigid transformation. If the difference in the protected position is greater than the first preset threshold, the target object undergoes a non-rigid transformation.

3. The method according to claim 1, characterized in that, The similarity is represented by the mutual information between the registered two-dimensional image and the pre-projected image; Based on the similarity, it is determined whether the deformation of the target object can be fitted by the two-dimensional transformation model, specifically including: If the mutual information is greater than or equal to the second preset threshold, then the deformation energy of the target object is fitted by the two-dimensional transformation model; If the mutual information is less than a second preset threshold, then the deformation of the target object cannot be fitted by the two-dimensional transformation model.

4. The method according to claim 1, characterized in that, The similarity is characterized by the grayscale difference between the registered two-dimensional image and the pre-projected image; Based on the similarity, it is determined whether the deformation of the target object can be fitted by the two-dimensional transformation model, specifically including: If the grayscale difference is less than or equal to a third preset threshold, then the deformation energy of the target object is fitted by the two-dimensional transformation model; If the grayscale difference is greater than a third preset threshold, then the deformation of the target object cannot be fitted by the two-dimensional transformation model.

5. The method according to claim 1, characterized in that, Converting the two-dimensional transformation relationship between the two-dimensional projected image and the previous projected image into a three-dimensional transformation relationship specifically includes: Set the rotation component in the three-dimensional transformation relationship as the identity matrix; Set the scaling component in the three-dimensional transformation relationship as the identity matrix; The horizontal translation component in the three-dimensional transformation relationship is set as the horizontal translation component in the two-dimensional transformation relationship. The translation component in the depth direction of the three-dimensional transformation relationship is set as the product of the scaling component in the two-dimensional transformation relationship and the distance from the X-ray source to the detector of the X-ray imaging device.

6. The method according to claim 1, characterized in that, The non-radioactive three-dimensional imaging device is an RGBD camera, and the target object is the spine. Using a non-radioactive 3D imaging device, 3D information of the target object's surface is acquired before surgery to generate the first point cloud, specifically including: The RGBD camera was used to acquire RGB and depth images of the patient's back before surgery; A semantic segmentation network is used to symmetrically segment the RGB image to obtain the left back region and the right back region. Calculate the midpoints of the boundaries of the left and right back regions in each row of the image or in each vertical slice along the spine, and perform curve fitting on all midpoints to obtain the surface marking lines of the spine. The spinal surface marking lines are fused with the depth image to generate the first point cloud.

7. The method according to claim 6, characterized in that, Before using a semantic segmentation network to perform symmetrical left-right segmentation on the RGB image, the method further includes: performing illumination normalization preprocessing on the RGB image.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 7.

9. A computer program product comprising a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 7.

Images