Body surface registration method, apparatus, device, and medium

Abstract

This application provides a method, apparatus, device, and medium for body surface registration. The method includes: calibrating multiple three-dimensional vision cameras using a three-dimensional calibration phantom to obtain a calibration transformation matrix between the camera coordinate system and the radiotherapy accelerator coordinate system corresponding to each three-dimensional vision camera; the three-dimensional calibration phantom includes a target number of markers, the target number being greater than or equal to 4, and the target number of markers being arranged in a multidimensional simplex structure; based on the calibration transformation matrix, transforming and fusing point cloud data obtained by scanning the patient's treatment area using multiple three-dimensional vision cameras to obtain original fused point cloud data; performing Laplacian pyramid filtering on the original fused point cloud data to obtain target fused point cloud data; and registering the patient's treatment position based on the original fused point cloud data, the target fused point cloud data, and the patient's treatment reference image. Using this spatially arranged three-dimensional calibration phantom helps improve calibration accuracy.

Description

Technical Field

[0001] This application relates to the field of radiotherapy technology, and more specifically, to a body surface registration method, apparatus, equipment, and medium. Background Technology

[0002] In Surface Guided Radiation Therapy (SGRT) systems, multiple cameras acquire patient surface data from different angles. This real-time data is then registered with reference surface data in the treatment plan, enabling precise monitoring and real-time adjustment of the patient's position. This method allows for continuous monitoring of the patient's surface morphology throughout the treatment process, enabling timely detection and correction of positioning deviations, significantly improving the accuracy and safety of radiotherapy.

[0003] To integrate the body surface data collected by various cameras into a unified coordinate system and establish a clear spatial correspondence with the coordinate system of the radiotherapy accelerator, camera calibration is required. The accuracy of the calibration directly affects the accuracy of subsequent registration. Most calibrations use a two-dimensional checkerboard calibration board, but this method requires a high shooting angle, is cumbersome, and has a large calibration error, thus affecting the calibration accuracy. Summary of the Invention

[0004] In view of this, this application provides a body surface registration method, apparatus, device and medium, which calibrates multiple three-dimensional vision cameras through a three-dimensional calibration phantom. The three-dimensional calibration phantom includes at least four markers, and the markers are arranged in a multi-dimensional simplex structure, which helps to improve the calibration accuracy.

[0005] Specifically, this application is implemented through the following technical solution:

[0006] According to a first aspect of this application, a body surface registration method is provided, the method comprising:

[0007] Multiple 3D vision cameras are calibrated using a 3D calibration phantom to obtain the calibration transformation matrix between the camera coordinate system and the radiotherapy accelerator coordinate system corresponding to each 3D vision camera; the 3D calibration phantom includes a target number of markers, the target number is greater than or equal to 4, and the target number of markers are arranged in a multidimensional simplex structure.

[0008] Based on the calibration transformation matrix, the point cloud data obtained by scanning the treatment area of ​​the patient's body surface through the multiple three-dimensional vision cameras is transformed and fused to obtain the original fused point cloud data.

[0009] The original fused point cloud data is subjected to Laplacian pyramid filtering to obtain the target fused point cloud data;

[0010] Based on the original fused point cloud data, the target fused point cloud data, and the patient's treatment reference image, the patient's treatment position is registered to determine the six degrees of freedom deviation between the patient's current position and the treatment reference position.

[0011] In one optional implementation, the calibration of multiple 3D vision cameras using a 3D calibration phantom to obtain a calibration transformation matrix between the camera coordinate system and the radiotherapy accelerator coordinate system corresponding to each 3D vision camera includes:

[0012] With the three-dimensional calibration phantom located at the center of the radiotherapy accelerator and leveled, images including the three-dimensional calibration phantom are acquired by the multiple three-dimensional vision cameras;

[0013] For each of the images acquired by the three-dimensional vision cameras, markers are detected, and the number of successfully detected markers is determined.

[0014] Based on the number of detections corresponding to each of the three-dimensional vision cameras, the calibration mode to be used is determined, and according to the calibration mode, for each of the three-dimensional vision cameras, based on the position of the marker in the radiotherapy accelerator coordinate system and the position in the camera coordinate system, the calibration transformation matrix between the camera coordinate system and the radiotherapy accelerator coordinate system corresponding to the three-dimensional vision camera is determined.

[0015] In one optional implementation, the step of determining the calibration mode to be used based on the number of detections corresponding to each of the three-dimensional vision cameras, and determining the calibration transformation matrix between the camera coordinate system and the radiotherapy accelerator coordinate system corresponding to each of the three-dimensional vision cameras according to the calibration mode, based on the position of the marker in the radiotherapy accelerator coordinate system and the position in the camera coordinate system, includes:

[0016] When the number of detections corresponding to each of the three-dimensional vision cameras is the same as the number of targets, the calibration mode used is determined to be the full inspection calibration mode. According to the full inspection calibration mode, for each of the three-dimensional vision cameras, based on the position of each marker in the radiotherapy accelerator coordinate system and the position of each marker extracted from the acquired image of the three-dimensional vision camera in the camera coordinate system, the calibration transformation matrix between the camera coordinate system and the radiotherapy accelerator coordinate system corresponding to the three-dimensional vision camera is determined.

[0017] If the number of detections corresponding to any 3D vision camera is 1 less than the target number, and the number of detections corresponding to other 3D vision cameras is greater than or equal to 1 less than the target number, then the calibration mode used is determined to be the fault-tolerant calibration mode. According to the fault-tolerant calibration mode, for each 3D vision camera, based on the position of each successfully detected marker in the radiotherapy accelerator coordinate system and the position of each successfully detected marker in the camera coordinate system extracted from the acquired image of the 3D vision camera, the calibration transformation matrix between the camera coordinate system and the radiotherapy accelerator coordinate system corresponding to the 3D vision camera is determined.

[0018] In one optional implementation, the step of performing Laplacian pyramid filtering on the original fused point cloud data to obtain the target fused point cloud data includes:

[0019] The original fused point cloud data is divided into multiple point cloud regions;

[0020] Based on the original fused point cloud data, the discrete point density of each point cloud region is determined, and voxel filtering is performed on each point cloud region according to the discrete point density of each point cloud region to obtain candidate fused point cloud data.

[0021] Based on the candidate fused point cloud data, the curvature of each point cloud region is determined, and each point cloud region is sampled according to the curvature of each point cloud region to obtain the preliminary fused point cloud data.

[0022] The target geometric features in the pre-fused point cloud data are enhanced to obtain the target fused point cloud data.

[0023] In one optional implementation, the step of performing voxel filtering on each point cloud region based on the discrete point density of each point cloud region to obtain candidate fused point cloud data includes:

[0024] For each point cloud region, a density threshold corresponding to the point cloud region is determined based on the discrete point density of the point cloud region.

[0025] Based on the comparison result between the density threshold corresponding to the point cloud region and the density of the discrete points, the voxel size is determined when performing voxel filtering on the point cloud region.

[0026] Voxel filtering is performed on each point cloud region according to the voxel size corresponding to each point cloud region to obtain the candidate fused point cloud data.

[0027] In one optional implementation, the step of sampling each point cloud region according to the curvature of each point cloud region to obtain pre-fused point cloud data includes:

[0028] For each point cloud region, a curvature threshold corresponding to the point cloud region is determined based on the local curvature of each discrete point in the point cloud region.

[0029] The sampling rate for downsampling the point cloud region is determined based on the comparison result between the curvature threshold corresponding to the point cloud region and the curvature corresponding to the point cloud region.

[0030] The point cloud regions are downsampled according to their respective sampling rates to obtain the pre-fused point cloud data.

[0031] In one optional implementation, the target geometric feature is determined through the following steps:

[0032] Based on the position of each discrete point and the distance between each discrete point, the curvature and normal vector of the surface formed by each discrete point are determined as the target geometric feature.

[0033] In one optional implementation, the step of registering the patient's treatment position based on the original fused point cloud data, the target fused point cloud data, and the patient's treatment reference image, and determining the six-degree-of-freedom deviation between the patient's current position and the treatment reference position, includes:

[0034] The current surface information is extracted from the original fused point cloud data and the target fused point cloud data;

[0035] Reference surface information was extracted from the treatment reference image of the patient;

[0036] Based on the improved iterative nearest point registration algorithm using surface information, the patient's treatment position is registered to determine the six-degree-of-freedom deviation between the patient's current position and the treatment reference position. Compared with the original iterative nearest point registration algorithm, the improved iterative nearest point registration algorithm using surface information adds feature extraction of the current surface information and the reference surface information.

[0037] According to a second aspect of this application, a body surface registration device is provided, the device comprising:

[0038] A camera calibration module is used to calibrate multiple three-dimensional vision cameras using a three-dimensional calibration phantom to obtain a calibration transformation matrix between the camera coordinate system and the radiotherapy accelerator coordinate system corresponding to each three-dimensional vision camera. The three-dimensional calibration phantom includes a target number of markers, the target number being greater than or equal to 4, and the target number of markers being arranged in a multidimensional simplex structure.

[0039] The point cloud conversion module is used to convert and fuse the point cloud data obtained by scanning the treatment area of ​​the patient's body surface through the multiple three-dimensional vision cameras based on the calibration transformation matrix, so as to obtain the original fused point cloud data.

[0040] The point cloud filtering module is used to perform Laplacian pyramid filtering on the original fused point cloud data to obtain the target fused point cloud data;

[0041] The position registration module is used to register the patient's treatment position based on the original fused point cloud data, the target fused point cloud data, and the patient's treatment reference image, and to determine the six-degree-of-freedom deviation between the patient's current position and the treatment reference position.

[0042] According to a third aspect of this application, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the body surface registration method described in the first aspect above.

[0043] According to a fourth aspect of this application, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the body surface registration method described in the first aspect.

[0044] The body surface registration method, apparatus, device, and medium provided in this application calibration method calibrates multiple three-dimensional vision cameras by using a three-dimensional calibration phantom with at least four markers arranged in a multi-dimensional simplex structure. Relying on the spatial layout characteristics of the three-dimensional calibration phantom, it can provide sufficient spatial constraints for solving the calibration transformation matrix between the camera coordinate system and the radiotherapy accelerator coordinate system, ensuring the unique determination of the three-dimensional spatial attitude. It also has excellent fault tolerance, and can still complete high-accuracy calibration even if any marker data is abnormal, which helps to improve calibration accuracy and robustness. Subsequently, combined with point cloud transformation and fusion, Laplacian pyramid filtering, and patient treatment position registration, the six-degree-of-freedom deviation between the patient's current position and the treatment reference position can be accurately determined, providing reliable support for the precise positioning of radiotherapy.

[0045] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of this disclosure.

[0046] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0047] Figure 1 This is a flowchart illustrating a body surface registration method according to an exemplary embodiment of this application;

[0048] Figure 2 This is a schematic diagram of a three-dimensional calibration phantom shown in an exemplary embodiment of this application;

[0049] Figure 3 This is a schematic diagram illustrating a process for determining the calibration mode to be used, as shown in an exemplary embodiment of this application;

[0050] Figure 4 This is a schematic diagram illustrating a Laplace pyramid filtering process according to an exemplary embodiment of this application;

[0051] Figure 5 This is a schematic diagram illustrating a body surface registration process according to an exemplary embodiment of this application;

[0052] Figure 6 This is a schematic diagram of a body surface registration device shown in an exemplary embodiment of this application;

[0053] Figure 7 This is a schematic diagram of the structure of a computer device shown in an exemplary embodiment of this application.

[0054] In the diagram: 20-3D calibration phantom; 21-marker; 22-base; 23-level ruler; 600-surface registration device; 601-camera calibration module; 602-point cloud conversion module; 603-point cloud filtering module; 604-position registration module; 700-computer equipment; 710-processor; 720-memory; 730-bus; 721-memory; 722-external memory. Detailed Implementation

[0055] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0056] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0057] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0058] Research has shown that radiotherapy is a crucial treatment for tumors, with its core objective being to precisely deliver high-energy rays to the tumor target area while maximizing the protection of surrounding healthy tissues. To achieve this, precise patient positioning during treatment is essential. Traditional positioning methods rely on surface marking, laser alignment, and image-guided techniques, which have limitations: surface marking and laser positioning systems only provide two-dimensional positional information of the marked points and cannot monitor real-time changes in the surface morphology of the entire treatment area; while image-guided techniques offer high precision, they involve additional radiation doses and cannot achieve continuous real-time monitoring.

[0059] The emergence of optically guided surface radiotherapy (SGRT) technology offers a solution to this problem. As a novel, non-contact, radiation-free patient positioning and monitoring technology, it has been widely used in clinical practice in recent years. The SGRT system uses multiple cameras to acquire patient surface data from different angles, registering the real-time acquired surface data with reference surface data in the treatment plan to achieve precise monitoring and real-time adjustment of the patient's position. This method allows for continuous monitoring of the patient's surface morphology throughout the treatment process, enabling timely detection and correction of positioning deviations, significantly improving the accuracy and safety of radiotherapy. SGRT systems have been widely applied in clinical settings such as deep breath-hold therapy for breast cancer, frameless stereotactic radiotherapy for intracranial tumors, and respiratory gating therapy for body tumors.

[0060] To fuse the surface data acquired by various cameras into a unified coordinate system and establish a clear spatial correspondence with the coordinate system of the radiotherapy accelerator, camera calibration is required. The accuracy of the calibration directly affects the accuracy of subsequent registration. One method is to attach reflective markers to the patient's surface and use an infrared camera to track the markers' positions. This method offers high accuracy but requires attaching markers to the patient's surface, affecting patient comfort, and the markers may fall off. Another method is to use a two-dimensional checkerboard calibration board. This method requires a high shooting angle and is cumbersome, requiring the calibration board to be in various different orientations within the camera's field of view to provide sufficient constraints; otherwise, the calibration error will significantly increase. For multi-camera joint calibration of the SGRT system, two-dimensional calibration requires an additional step to unify the coordinate systems of different cameras, which can amplify errors and affect the accuracy of subsequent image registration and multi-view fusion. The three-sphere calibration method can also be used. Although it can determine a unique three-dimensional plane, it lacks a redundancy verification mechanism. When the data of any sphere becomes unavailable due to environmental occlusion, detection anomalies, or other issues, the calibration process will fail and cannot be completed. At the same time, the constraints provided by the three spheres are limited, thus limiting the calibration accuracy.

[0061] Based on the above research, this application provides a body surface registration method. By using a three-dimensional calibration phantom with at least four markers arranged in a multidimensional simplex structure, multiple three-dimensional vision cameras are calibrated. Relying on the spatial layout characteristics of the three-dimensional calibration phantom, sufficient spatial constraints can be provided for solving the calibration transformation matrix between the camera coordinate system and the radiotherapy accelerator coordinate system, ensuring the unique determination of the three-dimensional spatial pose. It also has excellent fault tolerance, and can still complete high-accuracy calibration even if any marker data is abnormal, which helps to improve calibration accuracy and robustness.

[0062] To facilitate understanding of this embodiment, a detailed description of the body surface registration method disclosed in this application embodiment is provided first. The execution entity of the body surface registration method provided in this application embodiment is generally an electronic device with a certain computing power. This electronic device can be a server, which can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud storage, big data, and artificial intelligence platforms. In some possible implementations, this body surface registration method can be implemented by a processor calling computer-readable instructions stored in memory.

[0063] The following description, in conjunction with the accompanying drawings, illustrates a body surface registration method provided in an embodiment of this application.

[0064] See Figure 1 The diagram shown is a flowchart illustrating a body surface registration method according to an exemplary embodiment of this application. Figure 1 As shown in the figure, the body surface registration method provided in this embodiment includes steps S101 to S104, wherein:

[0065] S101: Multiple three-dimensional vision cameras are calibrated using a three-dimensional calibration phantom to obtain the calibration transformation matrix between the camera coordinate system and the radiotherapy accelerator coordinate system corresponding to each three-dimensional vision camera; the three-dimensional calibration phantom includes a target number of markers, the target number is greater than or equal to 4, and the target number of markers are arranged in a multidimensional simplex structure.

[0066] In this step, the three-dimensional calibration phantom is used to perform spatial calibration of multiple three-dimensional vision cameras, and the calibration transformation matrix between the camera coordinate system and the radiotherapy accelerator coordinate system corresponding to each three-dimensional vision camera is obtained, so as to establish a precise correspondence between the camera coordinate system and the radiotherapy accelerator coordinate system of each three-dimensional vision camera.

[0067] The radiation therapy linear accelerator is a core device for cancer radiotherapy. It uses high-energy X-rays or electron beams to precisely irradiate the tumor area, destroying cancer cell DNA and inhibiting their growth. The accelerator features precise positioning and controllable dosage, minimizing damage to surrounding normal tissues.

[0068] The specific number of the multiple 3D vision cameras is determined according to the actual needs of radiotherapy and is not limited here. The multiple 3D vision cameras are arranged in different locations within the treatment room to ensure that 3D point cloud data of the patient's body surface can be acquired from multiple angles. The specific spatial layout of the multiple 3D vision cameras is determined according to the actual needs of radiotherapy and is not limited here; it is sufficient to ensure comprehensive coverage of the treatment area on the patient's body surface.

[0069] Here, the three-dimensional calibration phantom includes a target number of markers arranged in a multidimensional simplex structure. In geometry, a simplex is a higher-dimensional generalization of triangles and tetrahedrons. An n-dimensional simplex consists of n+1 points in a general position (i.e., non-coplanar and non-collinear). For example, when the target number is 4, the 4 markers are arranged in a three-dimensional simplex structure. When the target number is 5, the 5 markers are arranged in a four-dimensional simplex structure. This spatial layout ensures a uniquely determined three-dimensional spatial orientation, while still providing a valid calibration reference even if any marker data is abnormal.

[0070] While ensuring redundancy, increasing the number of markers can improve calibration redundancy and robustness, as well as calibration accuracy and fault tolerance. However, increasing the number of markers makes the 3D calibration phantom structure more complex, increases manufacturing costs, and increases the possibility of mutual occlusion between markers. In practical applications, using four markers can better balance redundancy, cost, and ease of use.

[0071] For example, you can also refer to Figure 2 , Figure 2 This is a schematic diagram illustrating a three-dimensional calibration phantom as an exemplary embodiment of this application. Figure 2 As shown in the example, the 3D calibration phantom 20 includes four markers 21. In this example, the markers 21 are spheres. In other embodiments, the markers can be replaced with other easily detectable geometric features, such as cylinders, cones, or reflective marks of a specific shape, to ensure that the geometric features of the markers can be detected by the 3D vision camera and provide unique pose information in 3D space.

[0072] Optionally, the marker can be made of a material with optimized optical reflective properties, such as polytetrafluoroethylene, to generate a strong reflective signal during the scanning process of the 3D vision camera, so that the 3D vision camera can quickly and accurately capture the marker.

[0073] In some possible implementations, the three-dimensional calibration phantom 20 may further include a base 22, which may be made of a metal material, such as aluminum alloy, to ensure a stable structure. The markers 21 may be fixedly mounted on the base 22 to ensure the stability of the phantom and the long-term stability of the relative positions of the markers. In this example, the four markers 21 are non-coplanar and non-collinearly distributed in three-dimensional space. The four markers 21 are fixed on the base 22 to form a three-dimensional simplex structure. The lines connecting the centers of any three markers 21 can form a non-collinear triangle, used for fault-tolerant calibration when the data of any marker 21 is abnormal.

[0074] Optionally, the surface of the base 22 may be coated with a material that does not have optical reflective properties. For example, a matte paint may be sprayed on the surface of the base 22 to effectively prevent the base from generating reflective interference signals when scanned by a 3D vision camera, prevent the 3D vision camera from misidentifying the base area as a marker, reduce noise and artifacts in the point cloud data, and reduce the complexity of subsequent point cloud data processing.

[0075] In some possible implementations, the three-dimensional calibration model 20 may further include a level 23, which is disposed on the base 22 and can ensure that the three-dimensional calibration model 20 is placed in a horizontal state.

[0076] In some possible implementations, the calibration of multiple 3D vision cameras using a 3D calibration phantom to obtain a calibration transformation matrix between the camera coordinate system and the radiotherapy accelerator coordinate system for each 3D vision camera includes:

[0077] With the three-dimensional calibration phantom located at the center of the radiotherapy accelerator and leveled, images including the three-dimensional calibration phantom are acquired by the multiple three-dimensional vision cameras;

[0078] For each of the images acquired by the three-dimensional vision cameras, markers are detected, and the number of successfully detected markers is determined.

[0079] Based on the number of detections corresponding to each of the three-dimensional vision cameras, the calibration mode to be used is determined, and according to the calibration mode, for each of the three-dimensional vision cameras, based on the position of the marker in the radiotherapy accelerator coordinate system and the position in the camera coordinate system, the calibration transformation matrix between the camera coordinate system and the radiotherapy accelerator coordinate system corresponding to the three-dimensional vision camera is determined.

[0080] In the above steps, the three-dimensional calibration phantom can be placed at the center of the radiotherapy accelerator, ensuring that the center of the three-dimensional calibration phantom coincides with the center of the radiotherapy accelerator. The three-dimensional calibration phantom is then adjusted to a horizontal position using a level. Images including the three-dimensional calibration phantom can then be acquired using multiple three-dimensional vision cameras. Marker detection is performed on the acquired images from each of the three-dimensional vision cameras to determine the number of successfully detected markers. Optionally, to improve the accuracy of the detection count determination, the acquired images can be denoised after acquisition, and then marker detection can be performed on the denoised acquired images from each of the three-dimensional vision cameras.

[0081] Based on the number of detections corresponding to each of the three-dimensional vision cameras, a calibration mode is determined. Following this calibration mode, for each three-dimensional vision camera, based on the position of the marker in the radiotherapy accelerator coordinate system and its position in the camera coordinate system, a calibration transformation matrix between the camera coordinate system and the radiotherapy accelerator coordinate system is determined. It can be understood that the origin of the radiotherapy accelerator coordinate system is the isocenter of the radiotherapy accelerator. Since the three-dimensional calibration phantom is placed at the isocenter of the radiotherapy accelerator, the origin of the radiotherapy accelerator coordinate system is also the isocenter of the three-dimensional calibration phantom. Therefore, the position of the marker in the radiotherapy accelerator coordinate system can be determined.

[0082] In this way, by placing the 3D calibration phantom at the center of the radiotherapy accelerator and leveling it, a unified and accurate spatial reference is provided for camera calibration, effectively avoiding calibration errors introduced by phantom placement deviations. The calibration mode used is determined according to the number of detections, which can flexibly cope with scenarios where some marker detections fail, further enhancing the fault tolerance and adaptability of the calibration process. Based on the dual-end position data of the markers in the radiotherapy accelerator coordinate system and the camera coordinate system, the calibration transformation matrix is ​​determined through the spatial relationship of the markers, ensuring the accuracy and uniqueness of the transformation relationship between the two coordinate systems.

[0083] In some possible implementations, the step of determining the calibration mode to be used based on the number of detections corresponding to each of the three-dimensional vision cameras, and determining, according to the calibration mode, for each of the three-dimensional vision cameras, the calibration transformation matrix between the camera coordinate system and the radiotherapy accelerator coordinate system corresponding to the three-dimensional vision camera, based on the position of the marker in the radiotherapy accelerator coordinate system and the position in the camera coordinate system, includes:

[0084] When the number of detections corresponding to each of the three-dimensional vision cameras is the same as the number of targets, the calibration mode used is determined to be the full inspection calibration mode. According to the full inspection calibration mode, for each of the three-dimensional vision cameras, based on the position of each marker in the radiotherapy accelerator coordinate system and the position of each marker extracted from the acquired image of the three-dimensional vision camera in the camera coordinate system, the calibration transformation matrix between the camera coordinate system and the radiotherapy accelerator coordinate system corresponding to the three-dimensional vision camera is determined.

[0085] If the number of detections corresponding to any 3D vision camera is the target number minus 1, and the number of detections corresponding to other 3D vision cameras is greater than or equal to the target number minus 1, then the calibration mode used is determined to be the fault-tolerant calibration mode. According to the fault-tolerant calibration mode, for each 3D vision camera, based on the position of each successfully detected marker in the radiotherapy accelerator coordinate system and the position of each successfully detected marker in the camera coordinate system extracted from the acquired image of the 3D vision camera, the calibration transformation matrix between the camera coordinate system and the radiotherapy accelerator coordinate system corresponding to the 3D vision camera is determined.

[0086] For a clearer illustration of the calibration process, please refer to [link / reference needed]. Figure 3 , Figure 3 This is a schematic diagram illustrating a process for determining the calibration mode to be used, as shown in an exemplary embodiment of this application. Figure 3As shown, it is determined whether the number of detections corresponding to each 3D vision camera is greater than or equal to the target number minus 1. If so, and the number of detections corresponding to each 3D vision camera is the target number, then all markers have been successfully detected for each 3D vision camera. It can be determined that the calibration mode used is the full inspection calibration mode. According to the full inspection calibration mode, for each 3D vision camera, based on the position of each marker in the radiotherapy accelerator coordinate system and the position of each marker extracted from the acquired image of the 3D vision camera in the camera coordinate system, the calibration transformation matrix between the camera coordinate system and the radiotherapy accelerator coordinate system corresponding to the 3D vision camera is determined. If the number of detections corresponding to any 3D vision camera is one less than the number of targets, then a marker detection fails for that 3D vision camera. In this case, the calibration mode used is determined to be a fault-tolerant calibration mode. According to this fault-tolerant calibration mode, for each 3D vision camera, based on the positions of successfully detected markers in the radiotherapy accelerator coordinate system and the positions of successfully detected markers in the camera coordinate system extracted from the acquired images of the 3D vision camera, a calibration transformation matrix between the camera coordinate system and the radiotherapy accelerator coordinate system is determined. Otherwise, calibration is considered a failure, and a prompt message is issued to instruct adjustment of the 3D calibration mode position.

[0087] Thus, the camera calibration using the three-dimensional calibration mode of this embodiment has a redundancy mechanism. When all markers are visible, the full-inspection calibration mode is adopted, which can use the positions of all markers to determine the calibration transformation matrix. This fully utilizes the spatial constraint characteristics of the multidimensional simplex structure markers, ensuring the high accuracy and uniqueness of the transformation relationship between the camera coordinate system and the radiotherapy accelerator coordinate system. When any marker fails to be detected, the system automatically switches to the fault-tolerant calibration mode, which uses the positions of the successfully detected markers to determine the calibration transformation matrix. This effectively avoids the calibration failure problem caused by the failure of a single marker, significantly improving the fault tolerance and stability of the calibration process, and providing a reliable data foundation for the subsequent conversion of patient surface point cloud data.

[0088] S102: Based on the calibration transformation matrix, the point cloud data obtained by scanning the treatment area of ​​the patient's body surface through the multiple three-dimensional vision cameras is transformed and fused to obtain the original fused point cloud data.

[0089] In this step, point cloud data is obtained by simultaneously scanning the treatment area of ​​the patient's body surface from different angles using multiple 3D vision cameras. This point cloud data is in the camera coordinate system. For each 3D vision camera, the point cloud data in the camera coordinate system is transformed according to the calibration transformation matrix corresponding to the 3D vision camera to obtain point cloud data in the accelerator coordinate system. The point cloud data in the accelerator coordinate system corresponding to multiple 3D vision cameras are fused to obtain the original fused point cloud data. The original fused point cloud data can provide initial pose estimation for subsequent body surface registration.

[0090] S103: Perform Laplacian pyramid filtering on the original fused point cloud data to obtain the target fused point cloud data.

[0091] In this step, the point cloud data acquired by multiple 3D vision cameras typically reaches hundreds of thousands to millions of points, resulting in issues such as noise and redundancy. Before performing body surface registration, the original fused point cloud data is processed by Laplacian pyramid filtering to obtain the target fused point cloud data, thereby improving registration efficiency and accuracy.

[0092] In some possible implementations, the step of performing Laplacian pyramid filtering on the original fused point cloud data to obtain the target fused point cloud data includes:

[0093] The original fused point cloud data is divided into multiple point cloud regions;

[0094] Based on the original fused point cloud data, the discrete point density of each point cloud region is determined, and voxel filtering is performed on each point cloud region according to the discrete point density of each point cloud region to obtain candidate fused point cloud data.

[0095] Based on the candidate fused point cloud data, the curvature of each point cloud region is determined, and each point cloud region is sampled according to the curvature of each point cloud region to obtain the preliminary fused point cloud data.

[0096] The target geometric features in the pre-fused point cloud data are enhanced to obtain the target fused point cloud data.

[0097] To illustrate the filtering process more clearly, please refer to [link / reference needed]. Figure 4 , Figure 4 This is a schematic diagram illustrating a Laplace pyramid filtering process, as shown in an exemplary embodiment of this application. Figure 4The diagram illustrates a three-level processing flow for the original fused point cloud data. The original fused point cloud sequentially undergoes density-aware voxel filtering, feature-preserving sampling, and feature enhancement, ultimately outputting high-quality target fused point cloud data for registration. In the first level, the original fused point cloud data is divided into multiple point cloud regions. Here, each point cloud region has the same size, and the specific size of the point cloud region can be set according to filtering needs; no limitation is made here. Based on the original fused point cloud data, the discrete point density of each point cloud region is determined. Voxel filtering is then performed on each point cloud region based on its discrete point density to obtain candidate fused point cloud data. In the second level, based on the candidate fused point cloud data, the curvature of each point cloud region is determined. Each point cloud region is sampled based on its curvature to obtain preliminary fused point cloud data. In the third level, the target geometric features in the preliminary fused point cloud data are enhanced to obtain the target fused point cloud data.

[0098] Traditional multi-camera point cloud data processing suffers from a trade-off between processing efficiency and quality. SGRT systems employ multiple cameras, each with varying noise characteristics due to differences in position, angle, and lighting conditions. Traditional point cloud processing methods use a uniform noise model and filtering strategy, resulting in poor noise suppression. The amount of point cloud data generated by multiple cameras typically reaches hundreds of thousands to millions. While traditional voxel grid downsampling methods are fast, they fall short in preserving detailed features. Applying the same filtering and sampling strategies to different regions leads to excessive loss of information in feature regions or excessive redundant data in flat areas. A single downsampling strategy cannot simultaneously balance processing efficiency in flat areas and registration accuracy in feature regions. Data processing latency impacts real-time monitoring performance, making it difficult to meet the real-time requirements of SGRT systems.

[0099] In this embodiment, the original fused point cloud data is processed by regional fine-grained processing. By dividing the original fused point cloud data into multiple point cloud regions, targeted voxel filtering is first performed based on the discrete point density of each point cloud region. Then, targeted sampling is performed based on the curvature of each point cloud region. Next, the target geometric features are enhanced. This not only achieves efficient noise reduction and adaptive downsampling of massive point cloud data from multiple cameras, reducing the problem of low processing efficiency of massive point cloud data, but also accurately preserves the key geometric features required for registration through differentiated filtering and sampling. This ensures that the registration accuracy is not compromised while significantly improving the efficiency of point cloud data processing, resulting in target fused point cloud data that is both lightweight and highly recognizable. This provides high-quality data support for the accurate registration of subsequent patient treatment positions and effectively reduces registration deviations caused by point cloud data noise or feature blurring.

[0100] In other implementations, the number of Laplacian pyramid filtering layers can be adjusted according to actual needs, provided that the camera's frame rate is met. For scenarios with higher accuracy requirements, more filtering layers can be added for more refined feature preservation processing. Specifically, multiple layer groups can be added after the original third layer, with each layer group including the second and third layers.

[0101] In some possible implementations, the step of performing voxel filtering on each point cloud region based on the discrete point density of each point cloud region to obtain candidate fused point cloud data includes:

[0102] For each point cloud region, a density threshold corresponding to the point cloud region is determined based on the discrete point density of the point cloud region.

[0103] Based on the comparison result between the density threshold corresponding to the point cloud region and the density of the discrete points, the voxel size is determined when performing voxel filtering on the point cloud region.

[0104] Voxel filtering is performed on each point cloud region according to the voxel size corresponding to each point cloud region to obtain the candidate fused point cloud data.

[0105] In the above steps, for each point cloud region, a density threshold corresponding to the point cloud region can be determined based on the discrete point density of the point cloud region. The density threshold corresponding to the point cloud region is compared with the discrete point density to obtain a density comparison result. If the density comparison result indicates that the discrete point density is greater than the density threshold, the voxel size for voxel filtering of the point cloud region can be determined as a first voxel size. If the density comparison result indicates that the discrete point density is less than or equal to the density threshold, the voxel size for voxel filtering of the point cloud region can be determined as a second voxel size, where the second voxel size is smaller than the first voxel size. Voxel filtering is performed on each point cloud region according to the voxel size corresponding to each point cloud region to obtain the candidate fused point cloud data.

[0106] The specific values ​​of the first voxel size and the second voxel size can be determined according to the actual filtering needs, and are not limited here. For example, if the first voxel size is 10, the second voxel size will be 5 or 3.

[0107] Traditional voxel filtering divides the three-dimensional space into a uniform voxel grid. Although this method is fast, it uses a uniform voxel size for all regions, which cannot balance efficiency in flat areas and accuracy in feature regions.

[0108] In this embodiment, the voxel size is dynamically adjusted according to the local point cloud density. Smaller voxel sizes are used in sparse point cloud regions to retain more points, while larger voxel sizes are used in dense point cloud regions to improve processing efficiency. This achieves differentiated and adaptive control of voxel filtering parameters. While eliminating point cloud noise and reducing data processing load, it maximizes the preservation of the geometric feature integrity of the original fused point cloud, providing high-quality candidate fused point cloud data for subsequent curvature sampling and feature enhancement.

[0109] In some possible implementations, sampling each point cloud region based on its curvature to obtain pre-fused point cloud data includes:

[0110] For each point cloud region, a curvature threshold corresponding to the point cloud region is determined based on the local curvature of each discrete point in the point cloud region.

[0111] The sampling rate for downsampling the point cloud region is determined based on the comparison result between the curvature threshold corresponding to the point cloud region and the curvature corresponding to the point cloud region.

[0112] The point cloud regions are downsampled according to their respective sampling rates to obtain the pre-fused point cloud data.

[0113] In the above steps, for each point cloud region, the local curvature of each discrete point in the point cloud region can be determined, and the local curvature is used to indicate the degree of surface curvature corresponding to the discrete point. Based on the local curvature of each discrete point in the point cloud region, the curvature corresponding to the point cloud region is determined. Based on the mean and standard deviation of the local curvature of each discrete point in the point cloud region, the curvature threshold corresponding to the point cloud region is determined. The curvature threshold and the curvature corresponding to the point cloud region are compared to obtain a curvature comparison result. If the curvature comparison result indicates that the curvature is greater than the curvature threshold, the sampling rate for downsampling the point cloud region can be determined as a first sampling rate. If the curvature comparison result indicates that the curvature is less than or equal to the curvature threshold, the sampling rate for downsampling the point cloud region can be determined as a second sampling rate, where the second sampling rate is greater than the first sampling rate. Each point cloud region is downsampled according to the sampling rate corresponding to each point cloud region to obtain the pre-fused point cloud data.

[0114] The specific values ​​of the first sampling rate and the second sampling rate can be determined according to actual sampling needs, and are not limited here. For example, if the first sampling rate is 4%, the resulting second sampling rate is 20%.

[0115] Traditional sampling methods mostly employ uniform sampling, which sparsifies the point cloud according to a fixed spatial interval or ratio. Although this method is simple, it cannot adaptively adjust the sampling density based on local features.

[0116] In this embodiment, a smaller sampling rate is used for high curvature regions (such as the bridge of the nose, earlobes, shoulder contours, and other areas with obvious geometric features) to retain higher density points and avoid feature loss, thereby ensuring registration accuracy. For low curvature regions (such as the cheeks, forehead, chest, and other flat areas), a larger sampling rate is used to perform higher downsampling and data processing, thereby improving registration efficiency. While ensuring the integrity of the point cloud geometric features, the data processing efficiency is significantly improved, providing high-quality pre-fusion point cloud data for subsequent feature enhancement and body surface registration.

[0117] In some possible implementations, the target geometric features are determined through the following steps:

[0118] Based on the position of each discrete point and the distance between each discrete point, the curvature and normal vector of the surface formed by each discrete point are determined as the target geometric feature.

[0119] In this step, a surface can be constructed using a surface fitting algorithm based on the position of each discrete point and the distance between each discrete point, and then the curvature and normal vector of the surface can be determined. Here, the curvature can be the overall curvature of the surface and / or the local curvature of each point on the surface, and the normal vector can be the normal vector of each point on the surface.

[0120] In this way, by determining the curvature and normal vector of the surface based on the position and spacing of discrete points as the target geometric features, the core geometric parameters that characterize the morphology of the treatment area on the patient's body surface in the point cloud data can be accurately extracted. This provides a clear and key optimization object for subsequent feature enhancement processing, effectively ensuring the accurate representation of key body surface morphology by the target fused point cloud data and helping to improve the reliability of registration.

[0121] S104: Based on the original fused point cloud data, the target fused point cloud data, and the patient's treatment reference image, register the patient's treatment position and determine the six-degree-of-freedom deviation between the patient's current position and the treatment reference position.

[0122] In this step, the original fused point cloud data is used as the initial pose. Based on the original fused point cloud data, the target fused point cloud data, and the patient's treatment reference image, the patient's treatment position is registered to determine the six degrees of freedom deviation between the patient's current position and the treatment reference position, so as to monitor and adjust the patient's position.

[0123] In some possible implementations, the registration of the patient's treatment position based on the original fused point cloud data, the target fused point cloud data, and the patient's treatment reference image, and the determination of the six-degree-of-freedom deviation between the patient's current position and the treatment reference position, includes:

[0124] The current surface information is extracted from the original fused point cloud data and the target fused point cloud data;

[0125] Reference surface information was extracted from the treatment reference image of the patient;

[0126] Based on the improved iterative closest point registration algorithm using surface information, the patient's treatment position is registered to determine the six-degree-of-freedom deviation between the patient's current position and the treatment reference position. Compared with the original iterative closest point (ICP) registration algorithm, the improved iterative closest point registration algorithm using surface information adds feature extraction of the current surface information and the reference surface information.

[0127] In the above steps, registration parameters, such as the number of iterations and the iteration end threshold, can be determined based on the original fused point cloud data and the target fused point cloud data, using the iterative nearest point registration algorithm improved with surface information. According to these registration parameters, the patient's treatment position is registered using the iterative nearest point registration algorithm improved with surface information, determining the six-degree-of-freedom deviation between the patient's current position and the treatment reference position.

[0128] The six degrees of freedom include translation along the X / Y / Z axes and rotation around the X / Y / Z axes (pitch, roll, yaw).

[0129] In this way, camera calibration using the three-dimensional calibration phantom of this embodiment can improve calibration accuracy. The original fused point cloud data provides a reliable initial pose estimate for registration, allowing the registration algorithm to iterate from an initial value close to the true value, significantly reducing convergence time and the number of iterations by about 60%. While greatly shortening the registration time, it effectively avoids the risk of getting trapped in local optima. Furthermore, by introducing surface information into the iterative nearest point registration algorithm, it makes up for the limitation of the original algorithm relying only on point cloud position information, which helps to improve the accuracy and robustness of registration.

[0130] For example, see [link / reference] Figure 5 This is a schematic diagram illustrating a body surface registration process, as shown in an exemplary embodiment of this application. Figure 5As shown, camera calibration is first performed by calibrating multiple 3D vision cameras using a 3D calibration phantom to obtain the calibration transformation matrix between the camera coordinate system and the radiotherapy accelerator coordinate system for each 3D vision camera. Then, point cloud processing is performed. Based on the calibration transformation matrix, the point cloud data obtained by scanning the patient's treatment area using the multiple 3D vision cameras is transformed and fused to obtain raw fused point cloud data. This raw fused point cloud data is then subjected to Laplacian pyramid filtering to obtain target fused point cloud data. Next, surface registration is performed. Based on the raw fused point cloud data, the target fused point cloud data, and the patient's treatment reference image, the patient's treatment position is registered, determining the six-degree-of-freedom deviation between the patient's current position and the treatment reference position. Specific steps are described in the aforementioned embodiment and will not be repeated here.

[0131] To more clearly demonstrate the effects of the embodiments of this disclosure, calibration accuracy tests, point cloud filtering performance evaluations, and registration performance tests are performed, exemplarily. In the calibration accuracy test, the coordinates in the camera coordinate system captured by the 3D vision camera are calibrated against the world coordinates. The calibration results are used to transform the phantom coordinates in the camera coordinate system, and the deviation between the transformed results and the world coordinates is calculated. The calibration experiment is repeated multiple times. The results show that the calibration accuracy using the 3D calibration phantom in this embodiment is less than 0.3 mm, and the repeatability is less than 0.2 mm, significantly better than traditional calibration methods. In the calibration experiment, a situation where a single marker detection fails is artificially created by occluding the marker or changing the lighting to test the fault tolerance capability. The calibration accuracy in the fault-tolerant calibration mode is less than 0.5 mm, and the repeatability is less than 0.5 mm, both meeting the requirements for clinical applications.

[0132] In the performance evaluation of point cloud filtering, surface point cloud data of 20 patients (covering different body types and treatment sites) were collected. Traditional voxel raster filtering and the Laplacian pyramid filtering described in this embodiment were used for processing, and the processing time and registration accuracy were compared. The results showed that the traditional filtering method, while maintaining the same registration accuracy (approximately 0.5 mm), had a processing time of approximately 100 ms. When using larger voxels to speed up the processing to 45 ms, the registration accuracy decreased to approximately 1.5 mm. The Laplacian pyramid filtering described in this embodiment, with a processing time of only 40 ms, still maintained a registration accuracy of 0.3 mm, achieving a dual improvement in efficiency and accuracy.

[0133] In the registration performance test, multiple sets of human phantoms were used for registration testing in a clinical simulation environment. The test results show that the registration accuracy reached 0.3mm, and the single-frame processing time (from image acquisition to output of registration results) was less than 40ms. Compared with existing methods, the registration accuracy is improved by about 40%, and the processing time is reduced by about 60%, meeting the dual requirements of high accuracy and real-time performance of the SGRT system.

[0134] The body surface registration method provided in this application uses a three-dimensional calibration phantom with at least four markers arranged in a multidimensional simplex structure to calibrate multiple three-dimensional vision cameras. Relying on the spatial layout characteristics of the three-dimensional calibration phantom, it can provide sufficient spatial constraints for solving the calibration transformation matrix between the camera coordinate system and the radiotherapy accelerator coordinate system, ensuring the unique determination of the three-dimensional spatial pose. It also has excellent fault tolerance, and can still complete high-accuracy calibration even if any marker data is abnormal, which helps to improve calibration accuracy and robustness. Subsequently, combined with point cloud transformation and fusion, Laplacian pyramid filtering and patient treatment position registration, the six-degree-of-freedom deviation between the patient's current position and the treatment reference position can be accurately determined, providing reliable support for the precise positioning of radiotherapy.

[0135] Corresponding to the embodiments of the aforementioned body surface registration method, this application also provides embodiments of a body surface registration device.

[0136] Please see Figure 6 This is a schematic diagram illustrating a body surface registration device according to an exemplary embodiment of this application. Figure 6 As shown in the figure, the body surface registration device 600 provided in this application embodiment includes:

[0137] The camera calibration module 601 is used to calibrate multiple three-dimensional vision cameras through a three-dimensional calibration phantom to obtain the calibration transformation matrix between the camera coordinate system and the radiotherapy accelerator coordinate system corresponding to each three-dimensional vision camera; the three-dimensional calibration phantom includes a target number of markers, the target number is greater than or equal to 4, and the target number of markers are arranged in a multi-dimensional simplex structure.

[0138] Point cloud conversion module 602 is used to convert and fuse point cloud data obtained by scanning the treatment area of ​​the patient's body surface through the multiple three-dimensional vision cameras based on the calibration transformation matrix to obtain original fused point cloud data.

[0139] Point cloud filtering module 603 is used to perform Laplacian pyramid filtering on the original fused point cloud data to obtain target fused point cloud data;

[0140] The position registration module 604 is used to register the patient's treatment position based on the original fused point cloud data, the target fused point cloud data and the patient's treatment reference image, and determine the six degrees of freedom deviation between the patient's current position and the treatment reference position.

[0141] In some possible implementations, the camera calibration module 601 is specifically used for:

[0142] With the three-dimensional calibration phantom located at the center of the radiotherapy accelerator and leveled, images including the three-dimensional calibration phantom are acquired by the multiple three-dimensional vision cameras;

[0143] For each of the images acquired by the three-dimensional vision cameras, markers are detected, and the number of successfully detected markers is determined.

[0144] Based on the number of detections corresponding to each of the three-dimensional vision cameras, the calibration mode to be used is determined, and according to the calibration mode, for each of the three-dimensional vision cameras, based on the position of the marker in the radiotherapy accelerator coordinate system and the position in the camera coordinate system, the calibration transformation matrix between the camera coordinate system and the radiotherapy accelerator coordinate system corresponding to the three-dimensional vision camera is determined.

[0145] In some possible implementations, the camera calibration module 601, when determining the calibration mode to be used based on the number of detections corresponding to each of the three-dimensional vision cameras, and determining the calibration transformation matrix between the camera coordinate system and the radiotherapy accelerator coordinate system corresponding to each of the three-dimensional vision cameras according to the calibration mode, based on the position of the marker in the radiotherapy accelerator coordinate system and the position in the camera coordinate system, specifically is used for:

[0146] When the number of detections corresponding to each of the three-dimensional vision cameras is the same as the number of targets, the calibration mode used is determined to be the full inspection calibration mode. According to the full inspection calibration mode, for each of the three-dimensional vision cameras, based on the position of each marker in the radiotherapy accelerator coordinate system and the position of each marker extracted from the acquired image of the three-dimensional vision camera in the camera coordinate system, the calibration transformation matrix between the camera coordinate system and the radiotherapy accelerator coordinate system corresponding to the three-dimensional vision camera is determined.

[0147] If the number of detections corresponding to any 3D vision camera is the target number minus 1, and the number of detections corresponding to other 3D vision cameras is greater than or equal to the target number minus 1, then the calibration mode used is determined to be the fault-tolerant calibration mode. According to the fault-tolerant calibration mode, for each 3D vision camera, based on the position of each successfully detected marker in the radiotherapy accelerator coordinate system and the position of each successfully detected marker in the camera coordinate system extracted from the acquired image of the 3D vision camera, the calibration transformation matrix between the camera coordinate system and the radiotherapy accelerator coordinate system corresponding to the 3D vision camera is determined.

[0148] In some possible implementations, the point cloud filtering module 603 is specifically used for:

[0149] The original fused point cloud data is divided into multiple point cloud regions;

[0150] Based on the original fused point cloud data, the discrete point density of each point cloud region is determined, and voxel filtering is performed on each point cloud region according to the discrete point density of each point cloud region to obtain candidate fused point cloud data.

[0151] Based on the candidate fused point cloud data, the curvature of each point cloud region is determined, and each point cloud region is sampled according to the curvature of each point cloud region to obtain the preliminary fused point cloud data.

[0152] The target geometric features in the pre-fused point cloud data are enhanced to obtain the target fused point cloud data.

[0153] In some possible implementations, when the point cloud filtering module 603 performs voxel filtering on each point cloud region based on the discrete point density of each point cloud region to obtain candidate fused point cloud data, it is specifically used for:

[0154] For each point cloud region, a density threshold corresponding to the point cloud region is determined based on the discrete point density of the point cloud region.

[0155] Based on the comparison result between the density threshold corresponding to the point cloud region and the density of the discrete points, the voxel size is determined when performing voxel filtering on the point cloud region.

[0156] Voxel filtering is performed on each point cloud region according to the voxel size corresponding to each point cloud region to obtain the candidate fused point cloud data.

[0157] In some possible implementations, when the point cloud filtering module 603 samples each point cloud region according to the curvature of each point cloud region to obtain pre-fused point cloud data, it is specifically used for:

[0158] For each point cloud region, a curvature threshold corresponding to the point cloud region is determined based on the local curvature of each discrete point in the point cloud region.

[0159] The sampling rate for downsampling the point cloud region is determined based on the comparison result between the curvature threshold corresponding to the point cloud region and the curvature corresponding to the point cloud region.

[0160] The point cloud regions are downsampled according to their respective sampling rates to obtain the pre-fused point cloud data.

[0161] In some possible implementations, the point cloud filtering module 603 determines the target geometric features through the following steps:

[0162] Based on the position of each discrete point and the distance between each discrete point, the curvature and normal vector of the surface formed by each discrete point are determined as the target geometric feature.

[0163] In some possible implementations, the location registration module 604 is specifically used for:

[0164] The current surface information is extracted from the original fused point cloud data and the target fused point cloud data;

[0165] Reference surface information was extracted from the treatment reference image of the patient;

[0166] Based on the improved iterative nearest point registration algorithm using surface information, the patient's treatment position is registered to determine the six-degree-of-freedom deviation between the patient's current position and the treatment reference position. Compared with the original iterative nearest point registration algorithm, the improved iterative nearest point registration algorithm using surface information adds feature extraction of the current surface information and the reference surface information.

[0167] The specific implementation process of the functions and roles of each module in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0168] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0169] Based on the same technical concept, this application also provides a computer device 700, referring to... Figure 7 The diagram shown is a schematic representation of the structure of a computer device according to an exemplary embodiment of this application, comprising:

[0170] The processor 710, memory 720, and bus 730 are included. The memory 720 is used to store execution instructions and includes main memory 721 and external memory 722. The main memory 721, also known as internal memory, is used to temporarily store the operation data in the processor 710 and the data exchanged with external memory 722 such as hard disk. The processor 710 exchanges data with external memory 722 through main memory 721.

[0171] In this embodiment, the memory 720 is specifically used to store application code that executes the solution of this application, and its execution is controlled by the processor 710. That is, when the computer device 700 is running, the processor 710 communicates with the memory 720 through the bus 730, or the processor 710 communicates with the memory 720 through other means, so that the processor 710 executes the application code stored in the memory 720, and then performs the steps of the body surface registration method described in any of the foregoing embodiments.

[0172] The memory 720 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0173] Processor 710 may be an integrated circuit chip with signal processing capabilities. The aforementioned processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor.

[0174] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the computer device 700. In other embodiments of this application, the computer device 700 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0175] This disclosure also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the body surface registration method described in the above-described method embodiments. The storage medium may be a volatile or non-volatile computer-readable storage medium.

[0176] This disclosure also provides a computer program product storing a computer program. When the computer program is run by a processor, it executes the steps of the body surface registration method provided in any of the above embodiments of this disclosure. For details, please refer to the above method embodiments, which will not be repeated here.

[0177] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium, which can be a volatile or non-volatile computer-readable storage medium. In another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0178] Furthermore, embodiments of the subject matter and functional operation described in this specification can be implemented in the following ways: digital electronic circuits, tangibly embodied computer software or firmware, computer hardware including the structures disclosed in this specification and their structural equivalents, or combinations thereof. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory program carrier for execution by a data processing apparatus or for controlling the operation of a data processing apparatus. Alternatively or additionally, program instructions may be encoded on artificially generated propagation signals, such as machine-generated electrical, optical, or electromagnetic signals, which are generated to encode information and transmit it to a suitable receiving device for execution by the data processing apparatus. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or combinations thereof.

[0179] The processing and logic flow described in this specification can be executed by one or more programmable computers that execute one or more computer programs to perform corresponding functions by operating on input data and generating output. The processing and logic flow can also be executed by dedicated logic circuitry—such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits), and the device can also be implemented as dedicated logic circuitry.

[0180] Suitable computers for executing computer programs include, for example, general-purpose and / or special-purpose microprocessors, or any other type of central processing unit. Typically, the central processing unit receives instructions and data from read-only memory and / or random access memory. The basic components of a computer include a central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as disks, magneto-optical disks, or optical disks, or the computer will be operatively coupled to such mass storage devices to receive data from or transfer data to them, or both. However, a computer is not required to have such devices. Furthermore, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device such as a universal serial bus (USB) flash drive, to name a few.

[0181] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, such as semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks. Processors and memory may be supplemented by or incorporated into dedicated logic circuitry.

[0182] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily intended to describe features of specific embodiments of a particular invention. Certain features described in the various embodiments herein may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation thereof.

[0183] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0184] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings are not necessarily shown in a specific order or sequence to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.

[0185] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A body surface registration method, characterized by, The method includes: Multiple 3D vision cameras are calibrated using a 3D calibration phantom to obtain the calibration transformation matrix between the camera coordinate system and the radiotherapy accelerator coordinate system corresponding to each 3D vision camera; the 3D calibration phantom includes a target number of markers, the target number is greater than or equal to 4, and the target number of markers are arranged in a multidimensional simplex structure. Based on the calibration transformation matrix, the point cloud data obtained by scanning the treatment area of ​​the patient's body surface through the multiple three-dimensional vision cameras is transformed and fused to obtain the original fused point cloud data. The original fused point cloud data is subjected to Laplacian pyramid filtering to obtain the target fused point cloud data; Based on the original fused point cloud data, the target fused point cloud data, and the patient's treatment reference image, the patient's treatment position is registered, and the six-degree-of-freedom deviation between the patient's current position and the treatment reference position is determined. The calibration of multiple 3D vision cameras using a 3D calibration phantom to obtain the calibration transformation matrix between the camera coordinate system and the radiotherapy accelerator coordinate system for each 3D vision camera includes: With the three-dimensional calibration phantom located at the center of the radiotherapy accelerator and leveled, images including the three-dimensional calibration phantom are acquired by the multiple three-dimensional vision cameras; For each of the images acquired by the three-dimensional vision cameras, markers are detected, and the number of successfully detected markers is determined. Based on the number of detections corresponding to each of the three-dimensional vision cameras, the calibration mode to be used is determined, and according to the calibration mode, for each of the three-dimensional vision cameras, based on the position of the marker in the radiotherapy accelerator coordinate system and the position in the camera coordinate system, the calibration transformation matrix between the camera coordinate system and the radiotherapy accelerator coordinate system corresponding to the three-dimensional vision camera is determined. The step of determining the calibration mode to be used based on the number of detections corresponding to each of the three-dimensional vision cameras, and determining the calibration transformation matrix between the camera coordinate system and the radiotherapy accelerator coordinate system for each of the three-dimensional vision cameras according to the calibration mode, based on the position of the marker in the radiotherapy accelerator coordinate system and the position in the camera coordinate system, includes: When the number of detections corresponding to each of the three-dimensional vision cameras is the same as the number of targets, the calibration mode used is determined to be the full inspection calibration mode. According to the full inspection calibration mode, for each of the three-dimensional vision cameras, based on the position of each marker in the radiotherapy accelerator coordinate system and the position of each marker extracted from the acquired image of the three-dimensional vision camera in the camera coordinate system, the calibration transformation matrix between the camera coordinate system and the radiotherapy accelerator coordinate system corresponding to the three-dimensional vision camera is determined. If the number of detections corresponding to any 3D vision camera is 1 less than the target number, and the number of detections corresponding to other 3D vision cameras is greater than or equal to 1 less than the target number, then the calibration mode used is determined to be the fault-tolerant calibration mode. According to the fault-tolerant calibration mode, for each 3D vision camera, based on the position of each successfully detected marker in the radiotherapy accelerator coordinate system and the position of each successfully detected marker in the camera coordinate system extracted from the acquired image of the 3D vision camera, the calibration transformation matrix between the camera coordinate system and the radiotherapy accelerator coordinate system corresponding to the 3D vision camera is determined.

2. The method of claim 1, wherein, The step of performing Laplacian pyramid filtering on the original fused point cloud data to obtain the target fused point cloud data includes: The original fused point cloud data is divided into multiple point cloud regions; Based on the original fused point cloud data, the discrete point density of each point cloud region is determined, and voxel filtering is performed on each point cloud region according to the discrete point density of each point cloud region to obtain candidate fused point cloud data. Based on the candidate fused point cloud data, the curvature of each point cloud region is determined, and each point cloud region is sampled according to the curvature of each point cloud region to obtain the preliminary fused point cloud data. The target geometric features in the pre-fused point cloud data are enhanced to obtain the target fused point cloud data.

3. The method of claim 2, wherein, The step of performing voxel filtering on each point cloud region based on the discrete point density of each region to obtain candidate fused point cloud data includes: For each point cloud region, a density threshold corresponding to the point cloud region is determined based on the discrete point density of the point cloud region. Based on the comparison result between the density threshold corresponding to the point cloud region and the density of the discrete points, the voxel size is determined when performing voxel filtering on the point cloud region. Voxel filtering is performed on each point cloud region according to the voxel size corresponding to each point cloud region to obtain the candidate fused point cloud data.

4. The method of claim 2, wherein, The step of sampling each point cloud region based on the curvature of each point cloud region to obtain pre-fused point cloud data includes: For each point cloud region, a curvature threshold corresponding to the point cloud region is determined based on the local curvature of each discrete point in the point cloud region. The sampling rate for downsampling the point cloud region is determined based on the comparison result between the curvature threshold corresponding to the point cloud region and the curvature corresponding to the point cloud region. The point cloud regions are downsampled according to their respective sampling rates to obtain the pre-fused point cloud data.

5. The method of claim 2, wherein, The target geometric features are determined through the following steps: Based on the position of each discrete point and the distance between each discrete point, the curvature and normal vector of the surface formed by each discrete point are determined as the target geometric feature.

6. The method of claim 1, wherein, The process of registering the patient's treatment position based on the original fused point cloud data, the target fused point cloud data, and the patient's treatment reference image, and determining the six-degree-of-freedom deviation between the patient's current position and the treatment reference position, includes: The current surface information is extracted from the original fused point cloud data and the target fused point cloud data; Reference surface information was extracted from the treatment reference image of the patient; Based on the improved iterative nearest point registration algorithm using surface information, the patient's treatment position is registered to determine the six-degree-of-freedom deviation between the patient's current position and the treatment reference position. Compared with the original iterative nearest point registration algorithm, the improved iterative nearest point registration algorithm using surface information adds feature extraction of the current surface information and the reference surface information.

7. A body surface registration device, characterized by The device includes: A camera calibration module is used to calibrate multiple three-dimensional vision cameras using a three-dimensional calibration phantom to obtain a calibration transformation matrix between the camera coordinate system and the radiotherapy accelerator coordinate system corresponding to each three-dimensional vision camera. The three-dimensional calibration phantom includes a target number of markers, the target number being greater than or equal to 4, and the target number of markers being arranged in a multidimensional simplex structure. The point cloud conversion module is used to convert and fuse the point cloud data obtained by scanning the treatment area of ​​the patient's body surface through the multiple three-dimensional vision cameras based on the calibration transformation matrix, so as to obtain the original fused point cloud data. The point cloud filtering module is used to perform Laplacian pyramid filtering on the original fused point cloud data to obtain the target fused point cloud data; The position registration module is used to register the patient's treatment position based on the original fused point cloud data, the target fused point cloud data, and the patient's treatment reference image, and to determine the six-degree-of-freedom deviation between the patient's current position and the treatment reference position; The camera calibration module is specifically used for: With the three-dimensional calibration phantom located at the center of the radiotherapy accelerator and leveled, images including the three-dimensional calibration phantom are acquired by the multiple three-dimensional vision cameras; For each of the images acquired by the three-dimensional vision cameras, markers are detected, and the number of successfully detected markers is determined. Based on the number of detections corresponding to each of the three-dimensional vision cameras, the calibration mode to be used is determined, and according to the calibration mode, for each of the three-dimensional vision cameras, based on the position of the marker in the radiotherapy accelerator coordinate system and the position in the camera coordinate system, the calibration transformation matrix between the camera coordinate system and the radiotherapy accelerator coordinate system corresponding to the three-dimensional vision camera is determined. The camera calibration module, when determining the calibration mode to be used based on the number of detections corresponding to each of the three-dimensional vision cameras, and determining the calibration transformation matrix between the camera coordinate system and the radiotherapy accelerator coordinate system for each three-dimensional vision camera according to the calibration mode and based on the position of the marker in the radiotherapy accelerator coordinate system and the position in the camera coordinate system, specifically performs the following: When the number of detections corresponding to each of the three-dimensional vision cameras is the same as the number of targets, the calibration mode used is determined to be the full inspection calibration mode. According to the full inspection calibration mode, for each of the three-dimensional vision cameras, based on the position of each marker in the radiotherapy accelerator coordinate system and the position of each marker extracted from the acquired image of the three-dimensional vision camera in the camera coordinate system, the calibration transformation matrix between the camera coordinate system and the radiotherapy accelerator coordinate system corresponding to the three-dimensional vision camera is determined. If the number of detections corresponding to any 3D vision camera is 1 less than the target number, and the number of detections corresponding to other 3D vision cameras is greater than or equal to 1 less than the target number, then the calibration mode used is determined to be the fault-tolerant calibration mode. According to the fault-tolerant calibration mode, for each 3D vision camera, based on the position of each successfully detected marker in the radiotherapy accelerator coordinate system and the position of each successfully detected marker in the camera coordinate system extracted from the acquired image of the 3D vision camera, the calibration transformation matrix between the camera coordinate system and the radiotherapy accelerator coordinate system corresponding to the 3D vision camera is determined.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the body surface registration method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the body surface registration method according to any one of claims 1 to 6.

Images