Calibration phantom and method for determining a calibration phantom pose

Abstract

This application relates to the field of medical system coordinate system calibration technology, and provides a calibration phantom and a method for determining the pose of the calibration phantom. It allows for the calibration of image acquisition devices at different installation positions using different sets of calibration surfaces of the calibration phantom, without the need to rearrange the calibration phantom. The calibration phantom provided in this application includes multiple sets of calibration surfaces; within the same set of calibration surfaces, the first calibration surface is perpendicular to the second and third calibration surfaces respectively; the normal of the second calibration surface in the coordinate system of the image acquisition device forms a first angle with the target coordinate axis of the image acquisition device coordinate system; the normal of the third calibration surface in the coordinate system of the image acquisition device forms a second angle with the target coordinate axis; the relationship between the first and second angles is consistent across all sets of calibration surfaces.

Description

Technical Field

[0001] This application relates to the field of medical system coordinate system calibration technology, and in particular to a calibration phantom and a method for determining the pose of the calibration phantom. Background Technology

[0002] The optical surface-guided radiotherapy system contains two subsystems: a tracking system centered on an image acquisition device and a medical system, which can be an RT (Radiation Therapy) system centered on a radiotherapy head.

[0003] In order to accurately transmit the motion information of the radiotherapy subject tracked by the tracking system to the RT radiotherapy system, it is necessary to know in advance the transformation relationship between the coordinate system of the image acquisition device and the coordinate system of the medical system. This transformation relationship can be calculated and obtained through calibration technology.

[0004] When the tracking system includes multiple image acquisition devices installed in different locations, the calibration phantom needs to be rearranged to calibrate each image acquisition device, which takes a long time. Summary of the Invention

[0005] Therefore, it is necessary to provide a calibration phantom and a method, apparatus, computer device, storage medium, and computer program product for determining the pose of the calibration phantom in the coordinate system of the image acquisition device, in order to address the above-mentioned technical problems.

[0006] This application provides a calibration phantom, which includes multiple sets of calibration surfaces; in the same set of calibration surfaces, a first calibration surface is perpendicular to a second calibration surface and a third calibration surface, respectively.

[0007] The normal of the second calibration surface in the coordinate system of the image acquisition device forms a first angle with the target coordinate axis of the coordinate system of the image acquisition device; the normal of the third calibration surface in the coordinate system of the image acquisition device forms a second angle with the target coordinate axis.

[0008] The size relationship between the first included angle and the second included angle is consistent across all calibration surfaces.

[0009] In one embodiment, the first calibration surface, the second calibration surface, and the third calibration surface within the same group have different colors.

[0010] In one embodiment, the first calibration surfaces of different groups have the same color, the second calibration surfaces of different groups have the same color, and the third calibration surfaces of different groups have the same color.

[0011] In one embodiment, the surface dimensions of the calibration phantom are positively correlated with the target distance; the target distance is the distance between the installation location of the image acquisition device and the placement location of the calibration phantom.

[0012] In one embodiment, the calibration mold has three laser bonding slots; the laser bonding slots are used to bond lasers to position the calibration mold; the laser emission directions corresponding to the three laser bonding slots are perpendicular to each other.

[0013] In one embodiment, the first calibration surface of each group is the same surface on the calibration phantom, and the second calibration surface and the third calibration surface of each group are sequentially adjacent surfaces on the calibration phantom.

[0014] In one embodiment, the calibration phantom is a decahedron, and the surface corresponding to the first calibration face is an octagon.

[0015] In one embodiment, a set of calibration surfaces consists of three intersecting surfaces of the calibration phantom under the same image acquisition viewpoint.

[0016] This application provides a method for determining the pose of a calibration phantom in the coordinate system of an image acquisition device, the method comprising:

[0017] Multiple image acquisition devices are used to acquire point cloud data of the calibration phantom described in the above embodiments from their respective acquisition perspectives, thereby obtaining point cloud data obtained by each of the image acquisition devices.

[0018] The point cloud data obtained by each of the image acquisition devices is segmented and identified to determine the pose of the calibration phantom in the coordinate system of each image acquisition device.

[0019] In one embodiment, after determining the pose of the calibration phantom in the coordinate systems of each image acquisition device, the method includes:

[0020] Based on the pose of the calibration phantom in the coordinate systems of each image acquisition device and the pose of the calibration phantom in the coordinate system of the medical system, the transformation relationship between the coordinate systems of each image acquisition device and the coordinate system of the medical system is obtained.

[0021] This application provides a device for determining the pose of a calibration phantom, the device comprising:

[0022] The point cloud acquisition module is used to acquire point cloud data of the calibration phantom described in the above embodiments through multiple image acquisition devices from their respective acquisition perspectives, and to obtain the point cloud data obtained by each of the image acquisition devices.

[0023] The pose determination module is used to segment and identify the point cloud data obtained by each of the image acquisition devices, and determine the pose of the calibration phantom in the coordinate system of each image acquisition device.

[0024] This application provides a computer device, including a memory and a processor, wherein the memory stores a computer program and the processor executes the above-described method.

[0025] This application provides a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor using the methods described above.

[0026] This application provides a computer program product having a computer program stored thereon, the computer program being executed by a processor using the above-described method.

[0027] In the aforementioned calibration phantom and the method, apparatus, computer equipment, storage medium, and computer program products for determining the pose of the calibration phantom, the calibration phantom includes multiple sets of calibration surfaces. Therefore, different sets of calibration surfaces of the calibration phantom can be used to calibrate image acquisition devices at different installation positions without the need to rearrange the calibration phantom. Furthermore, within the same set of calibration surfaces, the first calibration surface is perpendicular to the second and third calibration surfaces, respectively. Based on this characteristic, multiple image acquisition devices can identify which point cloud surface corresponds to the first calibration surface from the point cloud data acquired from the corresponding set of calibration surfaces. In addition, the size relationship between the first included angle and the second included angle is consistent in each set of calibration surfaces. Based on this characteristic, multiple image acquisition devices can identify which point cloud surface corresponds to the second or third calibration surface from the point cloud data acquired from the corresponding set of calibration surfaces. In other words, when identifying which surface in the same set of calibration surfaces the point cloud surface acquired by each image acquisition device corresponds to, the above two characteristics are used, and the same set of segmentation and recognition algorithms can be used without the need to design different segmentation and recognition algorithms, thus improving calibration efficiency. Attached Figure Description

[0028] Figure 1 Here is a calibration scenario diagram from one embodiment;

[0029] Figure 2 This is a schematic diagram of point cloud acquisition performed by a camera in one embodiment;

[0030] Figure 3 This is a flowchart illustrating a method for determining the pose of a calibration phantom in the coordinate system of an image acquisition device in one embodiment.

[0031] Figure 4 This is a schematic diagram of the calibration process in one embodiment;

[0032] Figure 5(a) is a schematic diagram of the calibration process in one embodiment;

[0033] Figure 5(b) is a schematic diagram of the calibration process in one embodiment;

[0034] Figure 6 This is a structural block diagram of a device for determining the pose of a calibration phantom in the coordinate system of an image acquisition device, as shown in one embodiment.

[0035] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0037] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0038] The calibration phantom provided in this application can be applied to calibration scenarios for the coordinate systems of image acquisition devices and medical systems in tracking systems. The tracking system may include an image acquisition device such as a camera (hereinafter referred to as cam), and correspondingly, the coordinate system of the image acquisition device is called the camera coordinate system. The number of image acquisition devices included in the tracking system can be determined according to actual needs. In some scenarios, in order to acquire images of the left and right sides and the middle of the abdomen of the radiotherapy subject, the tracking system may include three image acquisition devices, such as... Figure 1 The images show cam1, cam2, and cam3.

[0039] Each image acquisition device has its own coordinate system, for example Figure 1 The diagram shows that cam1 has its own coordinate system x1-y1-z1, cam2 has its own coordinate system x2-y2-z2, and cam3 has its own coordinate system x3-y3-z3.

[0040] The image acquisition device can acquire point cloud data from the calibration phantom, and the acquired point cloud data is in its own coordinate system. For example, such as... Figure 2 As shown, cam1 performs point cloud acquisition on the calibration phantom, and the acquired point cloud data is located in the camera coordinate system x1-y1-z1 of cam1.

[0041] The calibration phantom provided in this application includes multiple sets of calibration surfaces; in the same set of calibration surfaces, the first calibration surface is perpendicular to the second calibration surface and the third calibration surface, respectively; the normal of the second calibration surface in the coordinate system of the image acquisition device forms a first angle with the target coordinate axis of the coordinate system of the image acquisition device; the normal of the third calibration surface in the coordinate system of the image acquisition device forms a second angle with the target coordinate axis; the size relationship between the first angle and the second angle is consistent in each set of calibration surfaces.

[0042] Taking a camera as an example for image acquisition: three intersecting surfaces under the same camera viewpoint are considered as a group. The calibration phantom includes multiple sets of calibration surfaces, characterizing that the calibration phantom has three intersecting surfaces under different camera viewpoints.

[0043] by Figure 1 Taking the calibration phantom shown as an example, the upper surface of the calibration phantom is denoted as i, and the side surfaces are denoted as a, b, c, d, e, f, and g, respectively. From the perspective of camera cam1, if cam1 can see three intersecting surfaces a, b, and i, then these three intersecting surfaces are considered a group. From the perspective of camera cam2, if cam2 can see three intersecting surfaces g, f, and i, then these three intersecting surfaces are considered a group. From the perspective of camera cam3, if cam3 can see three intersecting surfaces e, d, and i, then these three intersecting surfaces are considered a group. Therefore, the calibration phantom is said to include multiple groups of calibration surfaces. The number of groups can be determined according to the number of camera perspectives included in the tracking system. For example, when there are 3 camera perspectives, the calibration phantom includes 3 groups of calibration surfaces, with each camera perspective having its corresponding group.

[0044] Within the same group, one calibration plane is perpendicular to the other two calibration planes; this calibration plane is called the first calibration plane. Of the remaining two calibration planes, the one located to the left of the camera's line of sight is called the second calibration plane, and the one located to the right of the camera's line of sight is called the third calibration plane. For example, in the group of calibration planes {a, b, i}, i is perpendicular to a, i is perpendicular to b, and i is called the first calibration plane. Of the remaining two planes a and b, a is located to the left of cam1's line of sight, and a is called the second calibration plane; b is located to the right of cam1's line of sight, and b is called the third calibration plane.

[0045] In some embodiments, the first calibration surface may be the upper surface of the calibration phantom, and the second and third calibration surfaces may be the side surfaces of the calibration phantom, wherein the side surfaces of the calibration phantom may be rectangular; the size of the second calibration surface may be larger than the size of the third calibration surface.

[0046] In each set of calibration surfaces included in the calibration phantom, one calibration surface is perpendicular to the other two calibration surfaces. For example, in the three sets of calibration surfaces {a, b, i}, {g, f, i}, and {e, d, i}, there is a surface i that is perpendicular to the other two surfaces in the same set.

[0047] The angle between the normal of the second calibration surface in the coordinate system of the image acquisition device and the target coordinate axis of the coordinate system of the image acquisition device is the first angle; the angle between the normal of the third calibration surface in the coordinate system of the image acquisition device and the target coordinate axis is the second angle.

[0048] Taking a set of calibration surfaces {a, b, i} from the perspective of the cam1 camera as an example, if a is the second calibration surface, b is the third calibration surface, and the x1 axis of the cam1 camera coordinate system is the target coordinate axis, then the angle ∠a-x1 between a and x1 belongs to the first angle, and the angle ∠b-x1 between b and x1 belongs to the second angle.

[0049] Similarly, in a set of calibration surfaces {g, f, i} from the perspective of the cam2 camera, in this example, if g is the second calibration surface, f is the third calibration surface, and the x2 axis of the cam2 camera coordinate system is the target coordinate axis, then the angle ∠g-x2 between g and x2 belongs to the first angle, and the angle ∠f-x2 between f and x2 belongs to the second angle.

[0050] Similarly, in a set of calibration surfaces {e, d, i} from the perspective of the cam3 camera, in this example, if e is the second calibration surface, d is the third calibration surface, and the x3 axis of the cam3 camera coordinate system is the target coordinate axis, then the angle ∠e-x3 between e and x3 belongs to the first angle, and the angle ∠d-x3 between d and x3 belongs to the second angle.

[0051] The relationship between the first included angle and the second included angle is consistent across all calibration surfaces.

[0052] Taking the above three sets of calibration surfaces as examples, the size relationships between ∠a-x1 and ∠b-x1, ∠g-x2 and ∠f-x2, and ∠e-x3 and ∠d-x3 remain consistent. That is, ∠a-x1 is greater than ∠b-x1, ∠g-x2 is greater than ∠f-x2, and ∠e-x3 is greater than ∠d-x3, or ∠a-x1 is less than ∠b-x1, ∠g-x2 is less than ∠f-x2, and ∠e-x3 is less than ∠d-x3.

[0053] The aforementioned calibration phantom includes multiple sets of calibration surfaces. Therefore, different sets of calibration surfaces can be used to calibrate image acquisition devices at different installation positions without needing to rearrange the calibration phantom. Furthermore, within the same set of calibration surfaces, the first calibration surface is perpendicular to the second and third calibration surfaces, respectively. Based on this spatial geometric characteristic, multiple image acquisition devices can identify which point cloud surface corresponds to the first calibration surface from the point cloud data collected from the corresponding set of calibration surfaces. In addition, the size relationship between the first and second included angles is consistent across all sets of calibration surfaces. Based on this spatial geometric characteristic, multiple image acquisition devices can identify which point cloud surface corresponds to the second or third calibration surface from the point cloud data collected from the corresponding set of calibration surfaces. In other words, when identifying which surface in the same set of calibration surfaces the point cloud surface collected by each image acquisition device corresponds to, the above two characteristics can be used to employ the same segmentation and recognition algorithm, eliminating the need to design different segmentation and recognition algorithms and improving calibration efficiency.

[0054] In one embodiment, the first calibration surface, the second calibration surface, and the third calibration surface within the same group have different colors.

[0055] For example, in {a, b, i}, a is green, b is blue, and i is red; and in another example, in {g, f, i}, g is yellow, f is purple, and i is black.

[0056] In this embodiment, the first calibration surface, the second calibration surface, and the third calibration surface in the same group have different colors. Subsequently, point cloud segmentation and recognition can be performed based on color characteristics as a supplement to spatial geometric characteristics. When one type of information becomes invalid, the other type of information can still be used, thereby ensuring that the algorithm can run stably.

[0057] In one embodiment, the first calibration surfaces of different groups have the same color, the second calibration surfaces of different groups have the same color, and the third calibration surfaces of different groups have the same color.

[0058] For example, in {a, b, i}, {g, f, i} and {e, d, i}, i is the first calibration surface, and the color of each group of i is the same, which can be set to red. a, g and e belong to the second calibration surface and can all be set to green. b, f and d belong to the third calibration surface and can all be set to blue.

[0059] In this embodiment, the first calibration surface has the same color in all groups, the second calibration surface has the same color in all groups, and the third calibration surface has the same color in all groups. Therefore, when setting a point cloud segmentation and recognition algorithm based on color characteristics, fewer color parameters can be set, and the groups do not need to be considered. The same set of color parameters can be set for each group.

[0060] In one embodiment, the surface dimensions of the calibration phantom are positively correlated with the target distance; the target distance is the distance between the installation location of the image acquisition device and the placement location of the calibration phantom.

[0061] The surface dimensions of the calibration phantom can be determined based on the dimensions of each face of the calibration phantom. To improve calibration accuracy, the surface dimensions of the calibration phantom can be adaptively increased as the image acquisition device moves further away from the calibration phantom.

[0062] In this embodiment, the distance between the installation position of the image acquisition device and the placement position of the calibration phantom is related to the surface size of the calibration phantom. The greater the distance, the larger the surface size, so that the calibration phantom can occupy as much of the acquisition field of view of the image acquisition device as possible (such as the camera's field of view) and improve the calibration accuracy.

[0063] In one embodiment, the calibration mold has three laser bonding slots; the laser bonding slots are used to bond lasers to position the calibration mold; the laser emission directions corresponding to the three laser bonding slots are perpendicular to each other.

[0064] The positioning of the calibration phantom requires the use of a laser and laser bonding slots on the phantom. The laser can be a surface laser. Since the positioning of the calibration phantom is three-dimensional, the laser emits from three directions: x, y, and z, which are perpendicular to each other. Accordingly, there are three laser bonding slots on the calibration phantom: one for bonding the laser emitted from the x direction, one for bonding the laser emitted from the y direction, and one for bonding the laser emitted from the z direction.

[0065] When the three laser bonding slots can be aligned with their respective lasers, the setup is successful, and point cloud acquisition can then be performed.

[0066] In the above embodiments, three laser bonding grooves are provided on the calibration mold to bond the lasers emitted from three directions, thereby completing the positioning of the calibration mold in three-dimensional space and ensuring the accuracy of subsequent calibration.

[0067] In one embodiment, the first calibration surface of each group is the same surface on the calibration phantom, and the second and third calibration surfaces of each group are sequentially adjacent surfaces on the calibration phantom.

[0068] For example, in {a, b, i}, {g, f, i} and {e, d, i}, the first calibration surface of each group is the upper surface i of the calibration phantom; a and b in {a, b, i} are sequentially adjacent side surfaces on the calibration phantom, g and f in {g, f, i} are sequentially adjacent side surfaces on the calibration phantom, and e and d in {e, d, i} are sequentially adjacent side surfaces on the calibration phantom.

[0069] In the above embodiments, the first calibration surface of each group shares the same surface of the calibration phantom, reducing the geometric complexity of the calibration phantom. Furthermore, the second and third calibration surfaces of each group are sequentially adjacent surfaces on the calibration phantom, so that there are three intersecting surfaces under different camera views, which is conducive to point cloud segmentation and recognition.

[0070] In one embodiment, the calibration phantom is a decahedron, and the surface corresponding to the first calibration face is an octagon.

[0071] For example, in Figure 1 The calibration phantom shown includes ten surfaces: an upper surface i, a lower surface, and eight side surfaces. The upper surface i corresponds to the first calibration surface and is octagonal.

[0072] In the above embodiments, the calibration model is a decahedron, and the surface corresponding to the first calibration face is an octagon. When placing the model, there is no need to specifically identify the orientation of the calibration model, thus improving the placement efficiency of the calibration model.

[0073] In one embodiment, a set of calibration surfaces consists of three intersecting surfaces of the calibration phantom under the same image acquisition viewpoint.

[0074] Where the image acquisition device is a camera, the image acquisition perspective is the camera's perspective. Figure 1 Taking the calibration phantom shown as an example, the upper surface of the calibration phantom is denoted as i, and the side surfaces of the calibration phantom are denoted as a, b, c, d, e, f, and g, respectively. From the perspective of camera cam1, if camera cam1 can see three intersecting surfaces a, b, and i, then these three intersecting surfaces are considered a group; from the perspective of camera cam2, if camera cam2 can see three intersecting surfaces g, f, and i, then these three intersecting surfaces are considered a group; from the perspective of camera cam3, if camera cam3 can see three intersecting surfaces e, d, and i, then these three intersecting surfaces are considered a group.

[0075] In this embodiment, based on the image acquisition perspective, the intersecting surfaces of the calibration model are determined as a group to facilitate subsequent point cloud segmentation and recognition.

[0076] To better understand the above calibration phantom, an application example of the calibration phantom of this application is described in detail below.

[0077] The design process of a calibration phantom needs to consider various factors of the optical surface radiotherapy system, all of which will affect the final shape of the phantom. Before designing, it is necessary to determine the number of different camera angles. In some scenarios, the tracking system includes three camera angles, with one camera installed in each angle. These three camera angles are used to track the left, right, and middle parts of the abdomen of the radiotherapy subject, respectively. Therefore, the calibration phantom needs to design geometric information (such as spatial planes, spatial angles, and spatial topological relationships between surfaces) and color information (such as easily identifiable colors like red, green, and yellow surfaces) at least in three angles.

[0078] Then, depending on the type of laser beam, such as a surface laser, a laser fitting groove needs to be designed on the calibration phantom. When placing the calibration phantom, it needs to be moved so that the surface laser can fit perfectly into the laser fitting groove of the phantom. Since there are surface lasers emitted from three directions (x, y, and z directions) in the radiotherapy room, three laser fitting grooves can be set on the calibration phantom. When placing the calibration phantom, the three laser fitting grooves should fit the surface laser emitted from the corresponding direction.

[0079] Next, geometric and color information can be added to the calibration phantom. Geometric information facilitates the extraction of multiple geometric planes, enabling point cloud segmentation and recognition of different geometric surfaces, and is used to extract overall surface normal and corner information. Color information serves the same purpose as geometric information, using a different modality to ensure the complete extraction of multiple geometric planes, point cloud segmentation and recognition, and the extraction of surface normal and corner information. After adding the geometric and color information, a calibration phantom with three intersecting geometric color planes from different camera perspectives is obtained.

[0080] Considering that surface fitting and surface normal extraction require the use of surface point clouds, the surface size of the acquired calibration modality must be appropriate. The surface size of the calibration phantom is determined based on the distance between the camera mounting position and the surface of the calibration phantom. For example, if the distance between the camera mounting position and the surface of the calibration phantom increases, the surface size of the calibration phantom should be increased accordingly. The specific surface size of the calibration phantom depends on the FOV (Field of View) size captured by the camera and the actual clinical needs. An inappropriate surface size and placement distance of the calibration phantom will reduce the accuracy of surface fitting and surface normal extraction.

[0081] The final designed calibration phantom needs to undergo experimental testing to check its calibration robustness, calibration accuracy, and flexibility, and to determine whether it meets clinical needs (the error should be within the sub-millimeter level). If it does not meet the requirements, the current calibration phantom needs to be redesigned or iteratively modified until the requirements are met. At this point, the entire calibration phantom design is complete, resulting in a regular decahedron with symmetrical red octagons on the upper and lower surfaces, surrounded by eight side surfaces composed of small blue rectangular faces and large green rectangular faces arranged in a row, and with three laser bonding grooves.

[0082] This application also provides a method for determining the pose of a calibration phantom, the method comprising: Figure 3 The steps shown are as follows:

[0083] Step S301: Multiple image acquisition devices acquire point cloud data of the calibration phantom provided in other embodiments of this application from their respective acquisition perspectives, thereby obtaining point cloud data obtained by each image acquisition device.

[0084] The tracking system includes a camera as the image acquisition device, and the coordinate system of the image acquisition device is called the camera coordinate system. The number of camera viewpoints included in the tracking system can be determined according to actual needs. In some scenarios, to acquire data on the left, right, and middle parts of the abdomen of the radiotherapy subject, the tracking system may include three camera viewpoints. These three camera viewpoints are used to track the movement information of the left, right, and middle parts of the abdomen of the radiotherapy subject, respectively. Each camera viewpoint can be equipped with one camera. In this case, the tracking system includes three cameras, such as... Figure 1 The images show cam1, cam2, and cam3.

[0085] Each image acquisition device has its own coordinate system, for example Figure 1 The diagram shows that cam1 has its own coordinate system x1-y1-z1, cam2 has its own coordinate system x2-y2-z2, and cam3 has its own coordinate system x3-y3-z3.

[0086] The image acquisition device can acquire point cloud data from the calibration phantom, and the acquired point cloud data is in its own coordinate system. For example, such as... Figure 2 As shown, cam1 performs point cloud acquisition on the calibration phantom, and the acquired point cloud data is located in the camera coordinate system x1-y1-z1 of cam1.

[0087] Step S302: Segment and identify the point cloud data obtained by each image acquisition device to determine the pose of the calibration phantom in the coordinate system of each image acquisition device.

[0088] Taking the acquisition of the pose of the calibration phantom in one of the image acquisition devices as an example: After obtaining the point cloud data collected by the image acquisition device, the point cloud data can be segmented and identified; segmentation is mainly used to determine which points in the point cloud belong to the same surface, thus obtaining the point cloud surface; identification is mainly used to determine which of the three intersecting calibration surfaces the point cloud surface corresponds to.

[0089] After segmenting and recognizing the point cloud data, the point cloud surfaces corresponding to the three intersecting calibration surfaces seen by the image acquisition device from their respective acquisition perspectives can be determined. Then, the normal of the point cloud surface in the coordinate system of the image acquisition device is taken as the normal of the calibration surface corresponding to the point cloud surface in the coordinate system of the image acquisition device. Thus, the normals of the three intersecting calibration surfaces in the coordinate system of the image acquisition device are obtained.

[0090] The normals of the three intersecting calibration surfaces in the coordinate system of the image acquisition device can characterize the pose of the calibration phantom in the coordinate system of the image acquisition device (called the first pose).

[0091] The same algorithm can be used to determine the pose of the calibration phantom in the coordinate system of different image acquisition devices.

[0092] In the above method, multiple image acquisition devices acquire point clouds of the calibration phantom provided in other embodiments of this application from their respective acquisition perspectives. The point cloud data obtained by each image acquisition device can be obtained by using the same algorithm to segment and recognize the point cloud data obtained by each image acquisition device, determine the pose of the calibration phantom in the coordinate system of each image acquisition device, and improve the robustness of the point cloud segmentation and recognition algorithm.

[0093] In one embodiment, after determining the pose of the calibration phantom in each image acquisition device coordinate system, the method provided in this application further includes: obtaining the transformation relationship between each image acquisition device coordinate system and the medical system coordinate system based on the pose of the calibration phantom in each image acquisition device coordinate system and the pose of the calibration phantom in the medical system coordinate system.

[0094] Taking one of the image acquisition devices mentioned above as an example: the normals of the three intersecting calibration surfaces seen by the image acquisition device from its acquisition viewpoint in the medical system coordinate system can characterize the pose of the calibration phantom in the medical system coordinate system (called the second pose).

[0095] After obtaining the first pose, the image acquisition device coordinate system or the medical system coordinate system can be translated and rotated to match the first pose with the second pose. Based on the translation and rotation, the transformation relationship between the image acquisition device coordinate system and the medical system coordinate system is obtained to complete the calibration.

[0096] In the above embodiments, by using the pose of the calibration phantom in the coordinate system of each image acquisition device and the pose of the calibration phantom in the coordinate system of the medical system, the translation and rotation amounts are determined through the matching between the poses, thereby improving the accuracy of the transformation relationship between the coordinate systems of each image acquisition device and the coordinate system of the medical system.

[0097] To better understand the above method, an application example will be explained in detail below.

[0098] In this application example, the medical system is a radiotherapy system. The tracking system includes cameras as image acquisition devices, and correspondingly, the coordinate system of the image acquisition devices is called the camera coordinate system. The number of camera viewpoints included in the tracking system can be determined according to actual needs. In some scenarios, in order to acquire data on the left and right sides and the middle of the abdomen of the radiotherapy subject, the tracking system may include three camera viewpoints. These three camera viewpoints are used to track the movement information of the left and right sides and the middle of the abdomen of the radiotherapy subject, respectively. One camera can be installed in each camera viewpoint. In this case, the tracking system includes three cameras, such as... Figure 1 The images show cam1, cam2, and cam3.

[0099] Each camera has its own coordinate system, for example Figure 1 The diagram shows that cam1 has its own coordinate system x1-y1-z1, cam2 has its own coordinate system x2-y2-z2, and cam3 has its own coordinate system x3-y3-z3.

[0100] The camera can acquire point cloud data from the calibration phantom, and the acquired point cloud data is in its own coordinate system. For example, such as... Figure 2 As shown, cam1 performs point cloud acquisition on the calibration phantom, and the acquired point cloud data is located in the camera coordinate system x1-y1-z1 of cam1.

[0101] Three intersecting surfaces of a calibration phantom under the same camera view are considered as a group. The calibration phantom provided in this application example includes multiple groups of calibration surfaces, indicating that the calibration phantom has three intersecting surfaces under different camera viewpoints.

[0102] In the same group of calibration surfaces, the first calibration surface is perpendicular to the second and third calibration surfaces respectively; the angle between the normal of the second calibration surface in the coordinate system of the image acquisition device and the target coordinate axis of the coordinate system of the image acquisition device is the first angle; the angle between the normal of the third calibration surface in the coordinate system of the image acquisition device and the target coordinate axis is the second angle; the relationship between the first angle and the second angle is consistent in all groups of calibration surfaces.

[0103] by Figure 1Taking the calibration phantom shown as an example, the upper surface of the calibration phantom is denoted as i, and the side surfaces are denoted as a, b, c, d, e, f, and g, respectively. From the perspective of camera cam1, if cam1 can see three intersecting surfaces a, b, and i, then these three intersecting surfaces are considered a group. From the perspective of camera cam2, if cam2 can see three intersecting surfaces g, f, and i, then these three intersecting surfaces are considered a group. From the perspective of camera cam3, if cam3 can see three intersecting surfaces e, d, and i, then these three intersecting surfaces are considered a group. Therefore, the calibration phantom is said to include multiple groups of calibration surfaces. The number of groups can be determined according to the number of camera perspectives included in the tracking system. For example, when there are 3 camera perspectives, the calibration phantom includes 3 groups of calibration surfaces, with each camera perspective having its corresponding group.

[0104] Within the same group, one calibration plane is perpendicular to the other two calibration planes; this calibration plane is called the first calibration plane. Of the remaining two calibration planes, the one located to the left of the camera's line of sight is called the second calibration plane, and the one located to the right of the camera's line of sight is called the third calibration plane. For example, in the group of calibration planes {a, b, i}, i is perpendicular to a, i is perpendicular to b, and i is called the first calibration plane. Of the remaining two planes a and b, a is located to the left of cam1's line of sight, and a is called the second calibration plane; b is located to the right of cam1's line of sight, and b is called the third calibration plane.

[0105] The first calibration surface can be the upper surface of the calibration phantom, and the second and third calibration surfaces can be the adjacent side surfaces of the calibration phantom. The side surfaces of the calibration phantom can be rectangular. The size of the second calibration surface can be larger than the size of the third calibration surface.

[0106] In each set of calibration surfaces included in the calibration phantom, one calibration surface is perpendicular to the other two calibration surfaces. For example, in the three sets of calibration surfaces {a, b, i}, {g, f, i}, and {e, d, i}, there is a surface i that is perpendicular to the other two surfaces in the same set.

[0107] The angle between the normal of the second calibration surface in the camera coordinate system and the target coordinate axis in the camera coordinate system is the first angle; the angle between the normal of the third calibration surface in the camera coordinate system and the target coordinate axis is the second angle.

[0108] Taking a set of calibration surfaces {a, b, i} from the perspective of the cam1 camera as an example, if a is the second calibration surface, b is the third calibration surface, and the x1 axis of the cam1 camera coordinate system is the target coordinate axis, then the angle ∠a-x1 between a and x1 belongs to the first angle, and the angle ∠b-x1 between b and x1 belongs to the second angle.

[0109] Similarly, in a set of calibration surfaces {g, f, i} from the perspective of the cam2 camera, in this example, if g is the second calibration surface, f is the third calibration surface, and the x2 axis of the cam2 camera coordinate system is the target coordinate axis, then the angle ∠g-x2 between g and x2 belongs to the first angle, and the angle ∠f-x2 between f and x2 belongs to the second angle.

[0110] Similarly, in a set of calibration surfaces {e, d, i} from the perspective of the cam3 camera, in this example, if e is the second calibration surface, d is the third calibration surface, and the x3 axis of the cam3 camera coordinate system is the target coordinate axis, then the angle ∠e-x3 between e and x3 belongs to the first angle, and the angle ∠d-x3 between d and x3 belongs to the second angle.

[0111] The relationship between the first included angle and the second included angle is consistent across all calibration surfaces.

[0112] Taking the above three sets of calibration surfaces as examples, the size relationships between ∠a-x1 and ∠b-x1, ∠g-x2 and ∠f-x2, and ∠e-x3 and ∠d-x3 are kept consistent. That is, ∠a-x1 is greater than ∠b-x1, ∠g-x2 is greater than ∠f-x2, and ∠e-x3 is greater than ∠d-x3. At this time, the angle between the second calibration surface and the target coordinate axis is greater than the angle between the third calibration surface and the target coordinate axis.

[0113] Within the same group, the first, second, and third calibration surfaces have different colors; and the first calibration surfaces of different groups have the same color, the second calibration surfaces of different groups have the same color, and the third calibration surfaces of different groups have the same color.

[0114] For example, in {a, b, i}, {g, f, i} and {e, d, i}, i is the first calibration surface, and the color of each group of i is the same, which can be set to red. a, g and e belong to the second calibration surface and can all be set to green. b, f and d belong to the third calibration surface and can all be set to blue.

[0115] Among them, the surface size of the calibration phantom is positively correlated with the target distance; the target distance is the distance between the installation position of the image acquisition device and the placement position of the calibration phantom.

[0116] The surface dimensions of the calibration phantom can be determined based on the dimensions of each face of the calibration phantom. To improve calibration accuracy, the surface dimensions of the calibration phantom can be adaptively increased as the image acquisition device moves further away from the calibration phantom.

[0117] The calibration mold has three laser bonding slots; the laser bonding slots are used to bond the laser to position the calibration mold; the laser emission directions corresponding to the three laser bonding slots are perpendicular to each other.

[0118] When positioning the calibration phantom, a laser and laser bonding slots on the phantom are required. The laser can be a surface laser. Since the positioning of the calibration phantom is three-dimensional, the laser is emitted from three directions: x, y, and z, and these three laser emission directions are perpendicular to each other. Accordingly, there are three laser bonding slots on the calibration phantom: one for bonding the laser emitted from the x direction, one for bonding the laser emitted from the y direction, and one for bonding the laser emitted from the z direction.

[0119] When the three laser bonding slots can be aligned with their respective lasers, the setup is successful, and point cloud acquisition can then be performed.

[0120] Figure 1 The calibration phantom shown includes ten surfaces: an upper surface i, a lower surface, and eight side surfaces. The upper surface i corresponds to the first calibration surface, and both the upper and lower surfaces are octagonal. The eight side surfaces are composed of small blue rectangular surfaces and large green rectangular surfaces arranged sequentially, and each surface has three laser bonding grooves.

[0121] This application example uses the aforementioned calibration phantom to obtain the transformation relationship between the camera coordinate system and the radiotherapy system coordinate system. The following example uses the transformation relationship between the camera coordinate system of cam1 and the radiotherapy system coordinate system as an example to illustrate the specific calibration process. Figure 4 The steps shown are as follows:

[0122] Step S401: After cam1 is aligned with the three intersecting calibration surfaces of the calibration model, point cloud data of the calibration model is acquired through cam1; the point cloud data is in the camera coordinate system of cam1.

[0123] Step S402: When cam1 is an RGB-D camera (R is short for red; G is short for green; B is short for blue; D is short for depth), acquire the color image and depth image formed when cam1 acquires the point cloud data of the calibration phantom.

[0124] Step S403: Using one or two methods, segment and identify the point cloud data to determine the point cloud surface corresponding to each of the three intersecting calibration surfaces;

[0125] Step S404: Based on the normals of the point cloud surface in the camera coordinate system of cam1, obtain the normals of the three intersecting calibration surfaces in the camera coordinate system of cam1.

[0126] Step S405: Based on the normals of the three intersecting calibration surfaces in the camera coordinate system of cam1 and the normals in the radiotherapy system coordinate system, obtain the transformation relationship between the camera coordinate system of cam1 and the radiotherapy system coordinate system, and complete the coarse registration.

[0127] Step S406: Use the point cloud ICP (Iterative Closest Point) registration algorithm to complete the fine registration.

[0128] The following describes two available methods involved in step S403:

[0129] Available Method 1: Point cloud segmentation and recognition have a sequential order.

[0130] As shown in Figure 5(a), after cam1 is installed in the corresponding position, cam1 is positioned opposite calibration surfaces a, b, and i of the calibration model. Calibration surfaces a and b are side surfaces, calibration surface i is the top surface, and calibration surface i is perpendicular to calibration surfaces a and b. The angle between calibration surface a and the x-axis of the camera coordinate system of cam1 is greater than the angle between calibration surface b and the x-axis of the camera coordinate system of cam1.

[0131] In the point cloud acquisition stage, after acquiring the point cloud data of the calibration phantom through cam1, point cloud data located in the camera coordinate system of cam1 can be obtained. Next, in the point cloud segmentation stage, spatial plane fitting technology is used to segment the point cloud data to obtain three point cloud surfaces, which are also located in the camera coordinate system of cam1. Then, in the point cloud recognition stage, the three point cloud surfaces are identified to determine which of the calibration surfaces a, b, and i they correspond to.

[0132] In the recognition phase, specifically, the normals of the three point cloud surfaces in the camera coordinate system of cam1 can be obtained. Then, the angles between each pair of normals are calculated to determine that one normal is perpendicular to the other two. Combining this with prior knowledge of the calibration phantom, it can be determined that the point cloud surface corresponding to this normal corresponds to calibration surface i. Next, the angles between the other two normals and the x-axis can be calculated. Among these two normals, the normal with the larger angle to the x-axis is determined, and the point cloud surface corresponding to this normal corresponds to calibration surface a. The normal with the smaller angle to the x-axis is determined, and the point cloud surface corresponding to this normal corresponds to calibration surface b.

[0133] Method 2: There is no strict order between point cloud segmentation and recognition.

[0134] As shown in Figure 5(b), after cam1 is installed in the corresponding position, cam1 is positioned facing the calibration surfaces a, b, and i of the calibration phantom. The calibration surfaces a, b, and i are green, blue, and red, respectively.

[0135] During the point cloud acquisition stage, point cloud data of the calibration phantom in the camera coordinate system of cam1 can be obtained through point cloud acquisition, and corresponding color and depth images are obtained. Next, the point cloud segmentation and recognition stage is entered. Based on the colors in the color image, it can be determined that the green area is the area where calibration surface a is located, the blue area is the area where calibration surface b is located, and the red area is the area where calibration surface i is located. Then, the green, blue, and red areas are mapped to the depth image. In the point cloud data corresponding to the depth image, the point cloud of the green area is determined to obtain the corresponding point cloud surface, which corresponds to calibration surface a. Similarly, the point cloud of the blue area is determined to obtain the corresponding point cloud surface, which corresponds to calibration surface b. The point cloud of the red area is determined to obtain the corresponding point cloud surface, which corresponds to calibration surface i.

[0136] If step S403 uses two methods for segmentation and recognition, then two normals of the same calibration surface in the camera coordinate system of cam1 can be obtained. At this time, these two normals can be weighted and the weighted normals can be used for calibration.

[0137] It is understandable that the transformation relationship between the camera coordinate system and the radiotherapy system coordinate system of cam2 and cam3 can also be obtained in the above way.

[0138] This application example fully considers the integrity and complete automation of the calibration algorithm, providing both coarse and fine registration functions. The coarse and fine point cloud registrations are seamlessly integrated, automatically completing the entire calibration process without any manual intervention. Furthermore, this coarse registration algorithm utilizes only the common spatial geometric features of the calibration phantom, eliminating the need for prior information during installation and placement, or the design of complex geometric features for coarse registration. This ensures reliability and stability, preventing the introduction of errors and randomness. Additionally, this coarse registration algorithm exhibits high robustness, utilizing both the spatial geometric and color information of the calibration phantom. If one type of information fails, the other remains usable, guaranteeing algorithm stability. Moreover, this coarse registration algorithm boasts high accuracy. It simultaneously uses both spatial geometric and color information from the calibration phantom. If one type of information is inaccurate, the other can be cross-validated, fully leveraging both types of information to obtain more precise coarse registration data. This improves both the coarse registration accuracy and the overall accuracy of the registration algorithm. This registration algorithm has a very high degree of adaptability and can be directly used for calibration of multiple cameras installed from different angles. The operation is uniform and easy to use.

[0139] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0140] In one embodiment, such as Figure 6 As shown, a device for determining the pose of a calibration phantom is provided, comprising:

[0141] The point cloud acquisition module 601 is used to acquire point cloud data of the calibration phantom provided in other embodiments of this application through multiple image acquisition devices at their respective acquisition perspectives, and to acquire point cloud data obtained by each image acquisition device.

[0142] The pose determination module 602 is used to segment and identify the point cloud data obtained by each image acquisition device and determine the pose of the calibration phantom in the coordinate system of each image acquisition device.

[0143] In one embodiment, the apparatus further includes a transformation relationship acquisition module, configured to obtain the transformation relationship between the coordinate systems of each image acquisition device and the coordinate system of the medical system based on the pose of the calibration phantom in the coordinate systems of each image acquisition device and the pose of the calibration phantom in the coordinate system of the medical system.

[0144] Specific limitations regarding the device for determining the pose of the calibration phantom in the coordinate system of the image acquisition device can be found in the limitations of the method for determining the pose of the calibration phantom in the coordinate system of the image acquisition device mentioned above, and will not be repeated here. Each module in the aforementioned device for determining the pose of the calibration phantom in the coordinate system of the image acquisition device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independent of the processor in the computer device, or stored in software in the memory of the computer device, so that the processor can call and execute the operations corresponding to each module.

[0145] In one embodiment, a computer device is provided, the internal structure of which can be shown as follows: Figure 7As shown, the computer device includes a processor, memory, and network interface connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores data for determining the pose of the calibration phantom. The network interface communicates with external terminals via a network connection. The computer device also includes input / output interfaces (I / O interfaces), which are circuits connecting the processor and external devices for exchanging information; they are connected to the processor via a bus. When the computer program is executed by the processor, it implements a method for determining the pose of a calibration phantom.

[0146] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0147] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps in the various method embodiments described above.

[0148] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the various method embodiments described above.

[0149] In one embodiment, a computer program product is provided having a computer program stored thereon, the computer program being executed by a processor of the steps described in the various method embodiments above.

[0150] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0151] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0152] The above embodiments are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A calibration phantom, characterized in that, The calibration phantom includes multiple sets of calibration surfaces; in the same set of calibration surfaces, the first calibration surface is perpendicular to the second calibration surface and the third calibration surface, respectively, so as to identify the point cloud surface corresponding to the first standard surface in the point cloud data collected by the image acquisition device from the corresponding set of calibration surfaces; The normal of the second calibration surface in the coordinate system of the image acquisition device forms a first angle with the target coordinate axis of the coordinate system of the image acquisition device; the normal of the third calibration surface in the coordinate system of the image acquisition device forms a second angle with the target coordinate axis. The size relationship between the first included angle and the second included angle is consistent in each group of calibration surfaces, so as to identify the point cloud surface corresponding to the second calibration surface and the point cloud surface corresponding to the third calibration surface in the point cloud data collected by the image acquisition device from the corresponding group of calibration surfaces; The point cloud surface corresponding to the first standard surface, the point cloud surface corresponding to the second calibration surface, and the point cloud surface corresponding to the third calibration surface are used to determine the pose of the calibration phantom in the coordinate system of the image acquisition device.

2. The calibration phantom according to claim 1, characterized in that, The first calibration surface, the second calibration surface, and the third calibration surface in the same group have different colors.

3. The calibration phantom according to claim 2, characterized in that, The first calibration surfaces of different groups have the same color, the second calibration surfaces of different groups have the same color, and the third calibration surfaces of different groups have the same color.

4. The calibration phantom according to claim 1, characterized in that, The surface dimensions of the calibration phantom are positively correlated with the target distance; the target distance is the distance between the installation position of the image acquisition device and the placement position of the calibration phantom.

5. The calibration phantom according to claim 1, characterized in that, The calibration mold is provided with three laser bonding grooves; the laser bonding grooves are used to bond lasers to position the calibration mold; the laser emission directions corresponding to the three laser bonding grooves are perpendicular to each other.

6. The calibration phantom according to any one of claims 1 to 5, characterized in that, The first calibration surface of each group is the same surface on the calibration phantom, and the second calibration surface and the third calibration surface of each group are sequentially adjacent surfaces on the calibration phantom.

7. The calibration phantom according to claim 6, characterized in that, The calibration phantom is a decahedron, and the surface corresponding to the first calibration surface is an octagon.

8. The calibration phantom according to claim 1, characterized in that, A set of calibration surfaces consists of three intersecting surfaces of the calibration phantom under the same image acquisition viewpoint.

9. A method for determining the pose of a calibration phantom, characterized in that, The method includes: Multiple image acquisition devices perform point cloud acquisition on the calibration phantom according to any one of claims 1 to 8 from their respective acquisition perspectives, and obtain point cloud data obtained by each of the image acquisition devices. The point cloud data obtained by each of the image acquisition devices is segmented and identified to determine the pose of the calibration phantom in the coordinate system of each image acquisition device.

10. The method according to claim 9, characterized in that, After determining the pose of the calibration phantom in the coordinate systems of each image acquisition device, the method includes: Based on the pose of the calibration phantom in the coordinate systems of each image acquisition device and the pose of the calibration phantom in the coordinate system of the medical system, the transformation relationship between the coordinate systems of each image acquisition device and the coordinate system of the medical system is obtained.

Images