A 3D point cloud camera calibration method for radiotherapy rooms
By using cube-structured calibration objects and spatial constraint principles, the calibration process of 3D point cloud cameras in radiotherapy rooms is simplified, solving the cumbersome operation problem in existing technologies and realizing an efficient and convenient calibration method suitable for calibration scenarios of multiple cameras.
Patent Information
- Application Number
- CN202310224219.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-03-10
- Publication Date
- 2025-09-23
- Estimated Expiration
- 2043-03-10
Smart Images

Figure CN116205994B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of positioning patients for radiotherapy in radiotherapy rooms, and in particular to a 3D point cloud camera calibration method applied to radiotherapy rooms. Background Art
[0002] The Radiation Therapy Patient Positioning System is a three-dimensional (3D) imaging system based on stereoscopic vision technology. It is used to image the patient's skin surface in 3D before and during radiotherapy. It provides real-time visualization of the six-degree-of-freedom error between the patient's immediate skin surface and the standard surface after positioning, providing high-precision, real-time monitoring of the patient's skin movement during treatment.
[0003] In order to obtain complete and accurate three-dimensional surface information of the patient, the optical surface monitoring system needs to install 3D point cloud cameras at multiple positions and angles in the radiotherapy room to collect 3D surface information of the patient's skin surface at different angles. The multi-sensor information is fused through the point cloud to form a complete image, which is used for subsequent positioning of key positions on the patient's surface and monitoring of patient movement during radiotherapy.
[0004] Point cloud fusion requires first obtaining the external parameter calibration information of each sensor, that is, the transformation relationship between the coordinate system of each 3D point cloud camera itself and the unified world coordinate system. The world coordinate system is the coordinate system of the center of the radiotherapy room. For a certain radiotherapy room, this coordinate system is fixed. The coordinate system of the 3D point cloud camera itself is generally located at the center of the camera imaging, and is in a fixed position relative to the camera body, which is determined by the camera's inherent parameters.
[0005] 3D point cloud sensor calibration is a prerequisite for implementing the radiotherapy patient positioning system. The system will set up calibration and correction processes with different frequencies and targets, such as daily and monthly inspections. How to efficiently and accurately complete the 3D point cloud sensor calibration is crucial to the implementation of system functions and the effectiveness of the system.
[0006] In existing calibration methods, traditional 2D cameras usually use a checkerboard calibration plate for camera calibration. For 3D point cloud cameras, single or multiple spherical calibration objects can often be used for camera calibration.
[0007] When using a single spherical calibration object for camera calibration, it is often necessary to move the calibration object to a known specified position multiple times. The rotation and translation between different positions must meet certain constraints. For example, the rotation angle between different positions cannot be too small, and the calibration object must form a certain angle of rotation on the X, Y, and Z axes. This type of calibration process is relatively cumbersome and has certain requirements for on-site operation and the accuracy of the mobile calibration object device. However, for the use scenario of the radiotherapy patient positioning system, the operator is the on-site medical staff, and the calibration frequency is high, which does not meet the conditions for implementing this type of calibration scheme. Similarly, for the calibration scheme with multiple spherical calibration objects, this type of calibration scheme requires multiple calibration spheres to be integrated into a calibration device. The setting of the calibration spheres must meet certain spatial constraints. The device is relatively complex and large in size. It requires high calibration object processing accuracy and may also be difficult to implement. Therefore, it is crucial to design an accurate and efficient solution for 3D point cloud camera calibration of radiotherapy patient positioning systems. Summary of the Invention
[0008] In order to solve the problems existing in the prior art, the present invention provides a 3D point cloud camera calibration method applied to a radiotherapy room, comprising the following steps:
[0009] Based on inherent equipment in the radiotherapy room, a first initial position of a calibration object having a cubic structure is obtained;
[0010] Based on the first initial position, three 3D point cloud cameras are set, and a second initial position of a center point of a camera front surface of the 3D point cloud camera is determined according to the radiotherapy coordinate system isocenter;
[0011] According to the spatial constraint principle, the 3D point cloud camera is calibrated based on the first initial position and the second initial position to obtain the rotation and translation matrix between the radiotherapy room world coordinate system and the 3D point cloud camera coordinate system.
[0012] Preferably, in the process of setting up three 3D point cloud cameras, each 3D point cloud camera is marked, and the position of each 3D point cloud camera is adjusted according to the first initial position, wherein the ceiling is installed in three directions of 0°, 90° and 180° of the first initial position.
[0013] Preferably, in the process of obtaining the second initial position, the center point of the camera front surface of each 3D point cloud camera is adjusted to a position where the distance from the isocenter is 1.8 meters to obtain the second initial position.
[0014] Preferably, in the process of positioning the patient, based on the calibration object, three surfaces of the calibration object are fitted by any 3D point cloud camera, and plane equations of the three surfaces in the 3D point cloud camera coordinate system are constructed;
[0015] Based on the plane equation, by setting a coordinate point on each plane, according to the principle of spatial constraint, the spatial position coordinates of the coordinate points on different planes are obtained;
[0016] The spatial position coordinates are converted to the world coordinate system of the radiotherapy room through the rotation and translation matrix to realize the positioning of the 3D point cloud camera.
[0017] Preferably, in the process of constructing the plane equation, before constructing the plane equation, the plane equation construction elements are set, which are the minimum number of data points n, the plane error threshold e, the maximum iteration rounds R, and the number of interior point convergence N; the plane equation construction process is as follows:
[0018] Step 1: Randomly select n points to form the minimum data point set;
[0019] Step 2: Fit the plane equation using the least squares method on the point set;
[0020] Step 3: Substitute all points into the plane equation calculated in step 2 and calculate the number of inliers. Inliers are points that are located on the plane equation within the error threshold e. Suppose the plane equation is f(x) = 0, and inliers are points that are f(x) < e.
[0021] Step 4: Compare the number of interior points of the current plane equation and the previously obtained optimal plane equation, record the larger one as the current optimal plane equation, and record the parameters, interior point set, and number of interior points of the optimal plane equation;
[0022] Repeat steps 1-4 until the maximum iteration R is reached or the maximum number of inliers is greater than the number of inliers converged N.
[0023] Preferably, in the process of obtaining the spatial position coordinates, according to the three surfaces, the intersection of the three surfaces is obtained as the 0 point, and the first coordinate value of the 0 point in the 3D point cloud camera coordinate system is obtained;
[0024] According to the coordinate point on each surface and in accordance with the spatial constraint rule, obtaining a second coordinate value of the coordinate point in the 3D point cloud camera coordinate system;
[0025] The spatial position coordinates are generated according to the first coordinate value and the second coordinate value.
[0026] Preferably, in the process of obtaining the spatial position coordinates, according to the three surfaces, the intersection of the three surfaces is obtained as the 0 point, and the first coordinate value of the 0 point in the 3D point cloud camera coordinate system is obtained;
[0027] According to the coordinate points on each face, according to the spatial constraint rules, the second coordinate value of the coordinate point in the 3D point cloud camera coordinate system is obtained;
[0028] Generate spatial position coordinates according to the first coordinate value and the second coordinate value.
[0029] Preferably, in the process of obtaining the first coordinate value, the first coordinate value is expressed as:
[0030]
[0031] Among them, the coordinates of point 0 are (X0, y0, z0), and the plane equation parameters of surfaces A, B, and C are (A A , B A , C A , D A )、(A B , B B , C B , D B )、(A C , B C , C C , D C ).
[0032] Preferably, in the process of obtaining the second coordinate value, the second coordinate value is expressed as:
[0033]
[0034] Among them, the coordinates of point 1 are (x1, y1, z1), the coordinates of point 2 are (x2, y2, z2), and the coordinates of point 3 are (x3, y3, z3). W indicates that the distance between point 1 and point 0 is the side length of the calibration block.
[0035] Preferably, in the process of obtaining the rotation and translation matrix, the rotation and translation matrix is expressed as:
[0036]
[0037] Wherein, the subscript c represents the coordinate value in the camera coordinate system, and the subscript w represents the coordinate value in the world coordinate system of the radiotherapy room.
[0038] Preferably, the calibration system for implementing the calibration method includes:
[0039] a first data acquisition unit, configured to acquire a first initial position of a patient undergoing radiotherapy based on inherent equipment in the radiotherapy room;
[0040] a second data acquisition unit, configured to determine, based on the first initial position, a second initial position of a center point of a camera front surface of the 3D point cloud camera according to the three 3D point cloud cameras and the radiotherapy coordinate system isocenter;
[0041] The positioning module is used to obtain the rotation and translation matrix between the world coordinate system of the radiotherapy room and the coordinate system of the 3D point cloud camera, and calibrate the 3D point cloud camera according to the first initial position and the second initial position based on the spatial constraint principle.
[0042] The present invention discloses the following technical effects:
[0043] The present invention can fully cooperate with the arrangement of radiotherapy room equipment and monitoring system 3D cameras. It is simple, efficient, and easy to operate. The calibration work of all three cameras in the system can be completed by placing the calibration device once. The calibration algorithm effectively utilizes the constraint relationship between points, lines, and surfaces in the cube structure, and efficiently solves the transformation relationship between the world coordinate system of the radiotherapy room and the camera coordinate system to complete the system calibration. At the same time, for other similar calibration scenarios, such as multiple cameras placed symmetrically, the calibration device and calibration scheme designed by the invention have high reference value. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0045] Figure 1 This is a schematic diagram of a positioning device based on a radiation chamber according to an embodiment of the present invention;
[0046] Figure 2 Schematic diagram of a cubic calibration object designed for an embodiment of the present invention;
[0047] Figure 3 This is a schematic diagram of the design of a cubic calibration object with a side length of 100 mm according to an embodiment of the present invention;
[0048] Figure 4 Schematic diagram of the overall process of single 3D camera calibration according to the present invention;
[0049] Figure 5 Schematic diagram of the calibration method of the present invention. DETAILED DESCRIPTION
[0050] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of this application.
[0051] like Figure 1-5 As shown, embodiment 1: the present invention is installed and arranged as shown in the attached Figure 1 As shown in the figure, the system consists of three 3D point cloud cameras and one cube-shaped calibration object. The remaining devices such as the radiotherapy machine, treatment bed, and laser are inherent equipment in the radiotherapy room.
[0052] Figure 1 The center position (isocenter) of the radiotherapy room's world coordinate system can be determined and indicated by a laser pointer. The calibration system involves three 3D point cloud cameras, numbered 0, 1, and 2, mounted on the ceiling at 0°, 90°, and 180° from the treatment bed. The central axes of all three cameras must pass through the isocenter of the radiotherapy coordinate system, and the distance between the center point of the camera's front surface and the isocenter is 1.8 meters. After the ceiling is suspended, the lowest point of the camera is 2 meters above the horizontal ground. Mounting accuracy is not strictly required and does not affect the calibration system's accuracy.
[0053] The cubic calibration object designed for the calibration system is as shown in the attached Figure 2 As shown in the figure, there is no strict limit on the size of the calibration object, as long as the object size does not exceed the field of view of the 3D point cloud camera. Figure 3 The design of a cubic calibration object with a side length of 100mm is demonstrated. Specifically, the main body of the calibration object is a cube with a side length of 100mm. The material is ceramic and the color is white to enhance diffuse reflection, thereby improving the imaging quality of the 3D camera point cloud during calibration. The edge chamfer size is R2.5mm to increase the convenience of use and reduce damage to other objects and itself due to accidental collisions; the base of the calibration object is a flat rectangular structure, connected to the bottom of the calibration block body, to increase the stability of the entire calibration object and increase the convenience of packaging, transportation, and storage. It has a side length of 170mm, a height of 15mm, and is made of ceramic and black in color to enhance light absorption, thereby preventing the 3D point cloud camera from imaging, thereby reducing interference from useless information.
[0054] For the above equipment and layout, the calibration goal is to calculate the transformation relationship between the world coordinate system of the radiotherapy room and the 3D point cloud camera coordinate system, that is, the rotation and translation matrix RT.
[0055] For single 3D camera calibration, the overall process is as follows Figure 4 As shown in the figure, the calibration process for each camera is exactly the same.
[0056] Specifically, for each individual camera:
[0057] By shooting a frame of point cloud, we can obtain the point cloud data of the three faces of the cube calibration object, which are recorded as faces A, B, and C respectively. The plane equations of the three faces in the camera coordinate system are fitted and calculated. The algorithm flow is as follows:
[0058] Input: minimum number of data points n (n is greater than 3), plane error threshold e, maximum iteration rounds R, number of interior point convergence N;
[0059] 1. Randomly select n points to form the minimum data point set;
[0060] 2. Fit the plane equation using the least squares method on the point set;
[0061] 3. Substitute all points into the plane equation calculated in 2 and calculate the number of inliers. Inliers are points that lie on the plane equation within the error threshold e. Suppose the plane equation is f(x) = 0. Inliers are points that lie within the error threshold e.
[0062] 4. Compare the number of interior points of the current plane equation and the previously obtained optimal plane equation, record the larger one as the current optimal plane equation, and record the parameters, interior point set, and number of interior points of the optimal plane equation;
[0063] 5. Repeat steps 1-4 until the maximum iteration R is reached or the maximum number of inliers is greater than the number of inliers converged N.
[0064] After obtaining the plane equations of faces A, B, and C, the coordinates of point 0 can be obtained by calculating the intersection of the three faces. Let the coordinates of point 0 be (x0, y0, z0). The plane equation parameters of faces A, B, and C are (A A , B A , C A , D A )、(A B , B B , C B , D B )、(A C , B C , C C , D C ), then the three plane equations can be solved to obtain the coordinates of point 0 in the 3D point cloud camera coordinate system:
[0065]
[0066] Then, spatial constraints can be used to obtain the coordinate values of points 1, 2, and 3 in the 3D point cloud camera coordinate system. Taking point 1 as an example, let the coordinates of point 1 be (x1, y1, z1). Since point 1 is located on plane A and plane B, and the distance between point 1 and point 0 is the side length W of the calibration block (100mm), the following set of equations can be solved to obtain two sets of candidate coordinate values for point 1:
[0067]
[0068] One of the candidate values is the true value of point 1, and the other is a point symmetrical to point 1 about plane C. Since the 3D point cloud camera coordinate system is fixed, the true coordinates of point 1 can be obtained by the relative positions of point 0 and point 1. The screening conditions are:
[0069] x1<x0&z1<z0;
[0070] The same method can be used to obtain the coordinates of points 2 and 3 in the camera coordinate system. The specific equations for obtaining points 1 and 2 are as follows:
[0071]
[0072] Finally, the coordinates of points 0, 1, 2, and 3 in the camera coordinate system and the radiotherapy room world coordinate system are used to calculate the rotation and translation matrix between the two coordinate systems, completing the calibration. The coordinates in the 3D camera coordinate system are converted to the coordinates in the radiotherapy room world coordinate system using the following formula.
[0073]
[0074] Among them, the subscript c represents the coordinate value of the camera coordinate system, the subscript w represents the coordinate value in the world coordinate system of the radiotherapy room, and the coordinate values of points 0, 1, 2, and 3 in the world coordinate system of the radiotherapy room are determined by the placement of the calibration block. The placement and position acquisition can be easily completed by using the laser pointer in the radiotherapy room. Usually, the center of the world coordinate system of the radiotherapy room can be aligned with the center of the upper surface of the calibration block. The side length of the calibration cube is W. At this time, the coordinates of points 0, 1, 2, and 3 in the world coordinate system of the radiotherapy room are
[0075]
[0076] Three equations can be obtained for each point, and the unknown parameters in the rotation and translation matrix RT can be obtained by combining 4 groups of points and 12 equations.
[0077] At this point, a set of camera extrinsic calibration points has been completed. According to the design of the calibration device and calibration scheme in this article, the three sets of 3D cameras and the calibration device are designed and arranged symmetrically. Therefore, the same process as above can be used. The calibration process does not require any movement of the calibration equipment or human intervention. All 3D cameras can be calibrated using a unified algorithm.
[0078] At the same time, similar calibration scenarios, such as multi-angle and position 3D point cloud camera calibration, can be designed and implemented with reference to the scheme designed in the present invention.
[0079] Example 2: The present invention also provides a 3D point cloud camera calibration method for use in a radiotherapy room. This method does not require a calibration object, but directly calibrates the 3D point cloud camera using the patient, and includes the following steps:
[0080] Based on inherent equipment in the radiotherapy room, obtaining a first initial position of the patient undergoing radiotherapy;
[0081] Based on the first initial position, three 3D point cloud cameras are set, and a second initial position of a center point of a camera front surface of the 3D point cloud camera is determined according to the radiotherapy coordinate system isocenter;
[0082] Obtain the rotation and translation matrix between the radiotherapy room world coordinate system and the 3D point cloud camera coordinate system, and calibrate the 3D point cloud camera according to the first initial position and the second initial position based on the spatial constraint principle.
[0083] Preferably, in the process of setting up three 3D point cloud cameras, each 3D point cloud camera is marked, and the position of each 3D point cloud camera is adjusted according to the first initial position, wherein the ceiling is installed in three directions of 0°, 90° and 180° of the first initial position.
[0084] Preferably, in the process of obtaining the second initial position, the center point of the camera front surface of each 3D point cloud camera is adjusted to a position where the distance from the isocenter is 1.8 meters to obtain the second initial position.
[0085] Preferably, in the process of positioning the patient, a cube based on the patient is constructed, and three faces of the cube are fitted by any one of the 3D point cloud cameras, and plane equations of the three faces in the 3D point cloud camera coordinate system are constructed;
[0086] Based on the plane equation, a coordinate point is set on each plane. According to the principle of spatial constraint, the spatial position coordinates of the coordinate points on different planes are obtained. The coordinate points are used to represent the points where the patient's appearance contour coincides with each plane.
[0087] The spatial position coordinates are converted to the world coordinate system of the radiotherapy room through the rotation and translation matrix to realize the positioning of the 3D point cloud camera.
[0088] The present invention implements the method logic process of the 3D point cloud camera calibration method through a computer program, forms an executable program, and applies it to the control system of the radiotherapy room, so that the control system has the functional characteristics of rapid calibration of the 3D point cloud camera, providing technical support for the functional digital expansion of the radiotherapy room.
[0089] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.
[0090] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.
[0091] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
Claims
1. A 3D point cloud camera calibration method used in radiotherapy rooms, characterized in that: The following steps are involved: Based on inherent equipment in the radiotherapy room, obtaining a first initial position of the patient undergoing radiotherapy; Based on the first initial position, three 3D point cloud cameras are set, and a second initial position of a center point of a camera front surface of the 3D point cloud camera is determined according to the radiotherapy coordinate system isocenter; Obtain the rotation and translation matrix between the radiotherapy room world coordinate system and the 3D point cloud camera coordinate system, and calibrate the 3D point cloud camera based on the first initial position and the second initial position according to the spatial constraint principle; In the process of setting up three 3D point cloud cameras, each 3D point cloud camera is marked, and the position of each 3D point cloud camera is adjusted according to the first initial position, wherein the 3D point cloud camera is ceiling-mounted in three directions of 0°, 90°, and 180° of the first initial position; During the patient positioning process, a cube based on the patient is constructed. The three faces of the cube are fitted using any 3D point cloud camera, and the plane equations of the three faces in the 3D point cloud camera coordinate system are constructed. Based on the plane equation, a coordinate point is set on each plane, and according to the principle of spatial constraint, the spatial position coordinates of the coordinate points on different planes are obtained, wherein the coordinate points are used to represent the coincidence points between the patient's appearance contour and each plane; The spatial position coordinates are converted to the world coordinate system of the radiotherapy room through the rotation and translation matrix to realize the positioning of the 3D point cloud camera.
2. The 3D point cloud camera calibration method for a radiotherapy room according to claim 1, characterized in that: In the process of obtaining the second initial position, the center point of the camera front surface of each of the 3D point cloud cameras is adjusted to a position where the distance from the isocenter is 1.8 meters to obtain the second initial position.
3. The 3D point cloud camera calibration method for a radiotherapy room according to claim 1, characterized in that: In the process of constructing the plane equation, before constructing the plane equation, the plane equation construction elements are set, which are the minimum number of data points n, the plane error threshold e, the maximum iteration rounds R, and the number of interior point convergence N. The plane equation construction process is as follows: Step 1: Randomly select n points to form the minimum data point set; Step 2: Fit the plane equation using the least squares method on the point set; Step 3: Substitute all points into the plane equation calculated in step 2 and calculate the number of inliers. Inliers are points that are located on the plane equation within the error threshold e. Suppose the plane equation is f(x) = 0, and inliers are points that are f(x) < e. Step 4: Compare the number of interior points of the current plane equation and the previously obtained optimal plane equation, record the larger one as the current optimal plane equation, and record the parameters, interior point set, and number of interior points of the optimal plane equation; Repeat steps 1-4 until the maximum iteration R is reached or the maximum number of inliers is greater than the number of inliers converged N.
4. The 3D point cloud camera calibration method for a radiotherapy room according to claim 3, characterized in that: In the process of obtaining the spatial position coordinates, according to the three faces, the intersection of the three faces is obtained as the 0 point, and the first coordinate value of the 0 point in the 3D point cloud camera coordinate system is obtained; According to the coordinate point on each surface and in accordance with the spatial constraint rule, obtaining a second coordinate value of the coordinate point in the 3D point cloud camera coordinate system; The spatial position coordinates are generated according to the first coordinate value and the second coordinate value.
5. The 3D point cloud camera calibration method for a radiotherapy room according to claim 4, characterized in that: In the process of obtaining the first coordinate value, the first coordinate value is expressed as: Among them, the coordinates of point 0 are (x0, y0, z0), and the plane equation parameters of surfaces A, B, and C are (A A , B A , C A , D A )、(A B , B B , C B , D B )、(A C , B C , C C , D C ).
6. The 3D point cloud camera calibration method for a radiotherapy room according to claim 5, characterized in that: In the process of obtaining the second coordinate value, the second coordinate value is expressed as: Among them, the coordinates of point 1 are (x1, y1, z1), the coordinates of point 2 are (x2, y2, z2), and the coordinates of point 3 are (x3, y3, z3). W indicates that the distance between point 1 and point 0 is the side length of the calibration block.
7. The 3D point cloud camera calibration method for a radiotherapy room according to claim 6, characterized in that: In the process of obtaining the rotation and translation matrix, the rotation and translation matrix is expressed as: Wherein, the subscript c represents the coordinate value in the camera coordinate system, and the subscript w represents the coordinate value in the world coordinate system of the radiotherapy room.
Citation Information
Patent Citations
Method and device for calibrating multiple 3D cameras based on polyhedron geometric constraint
CN114549651A
Three-dimensional pattern for camera calibration
JP2013231900A