Method, device and equipment for converting medical system coordinate system

Abstract

The application relates to the technical field of medical system coordinate system processing, and provides a medical system coordinate system conversion method, device and equipment, which have high robustness. In the application, point cloud data of a calibration model is collected by an image acquisition device at positions of three intersecting calibration surfaces of the calibration model; the point cloud data is segmented to obtain three point cloud surfaces, and the three intersecting calibration surfaces each correspond to a point cloud surface in the three point cloud surfaces; the normal lines of the three intersecting calibration surfaces in an image acquisition device coordinate system are obtained according to the point cloud surfaces corresponding to the three intersecting calibration surfaces respectively and the normal lines of the three point cloud surfaces in the image acquisition device coordinate system, and the conversion relationship between the image acquisition device coordinate system and a medical system coordinate system is obtained in combination with the normal lines of the three intersecting calibration surfaces in the medical system coordinate system.

Description

Technical Field

[0001] This application relates to the field of medical system coordinate system processing technology, and in particular to a method, apparatus, computer equipment, storage medium and computer program product for medical system coordinate system transformation. Background Technology

[0002] The optical surface-guided radiotherapy system comprises two subsystems: a tracking system centered on an image acquisition device and a medical system. This medical system can be an RT (Radiation Therapy) system centered on a radiotherapy head. During the operation of the RT system, the tracking system is responsible for tracking the surface of the patient and transmitting the patient's real-time motion information to the RT system using coordinate transformations acquired before treatment. After receiving the latest motion data, the RT system can promptly correct the target area for more precise radiation and improved treatment accuracy.

[0003] To accurately transmit motion information from the tracking system to the RT radiotherapy system, the transformation relationship between the image acquisition device's coordinate system and the medical system's coordinate system needs to be known in advance. This transformation relationship can be calculated using calibration techniques. However, traditional calibration techniques have relatively low robustness. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, apparatus, computer equipment, storage medium, and computer program product for converting coordinate systems in a medical system, in order to address the aforementioned technical problems.

[0005] This application provides a method for transforming a coordinate system in a medical system, the method comprising:

[0006] At the target location, point cloud data of the calibration phantom is acquired using an image acquisition device; the target location is the position of three intersecting calibration surfaces of the calibration phantom.

[0007] The point cloud data is segmented to obtain three point cloud surfaces. Among the three point cloud surfaces, the point cloud surface corresponding to each of the three intersecting calibration surfaces is determined.

[0008] Based on the point cloud surfaces corresponding to the three intersecting calibration surfaces, and the normals of the three point cloud surfaces in the coordinate system of the image acquisition device, three target normals are obtained.

[0009] Based on the three target normals and the normals of the three intersecting calibration surfaces in the medical system coordinate system, the transformation relationship between the image acquisition device coordinate system and the medical system coordinate system is obtained.

[0010] In one embodiment, the point cloud data is segmented to obtain three point cloud surfaces. Among these three point cloud surfaces, the point cloud surface corresponding to each of the three intersecting calibration surfaces is determined, including:

[0011] The point cloud data is segmented and identified to determine the first type of point cloud surface corresponding to each of the three intersecting calibration surfaces and the second type of point cloud surface corresponding to each of the three intersecting calibration surfaces.

[0012] In one embodiment, three target normals are obtained based on the point cloud surfaces corresponding to the three intersecting calibration surfaces and the normals of the three point cloud surfaces in the image acquisition device coordinate system, including:

[0013] Based on the first type point cloud surface corresponding to each of the three intersecting calibration surfaces, and the normals of the three first type point cloud surfaces, the first type normals of each of the three intersecting calibration surfaces are obtained.

[0014] Based on the second type point cloud surface corresponding to each of the three intersecting calibration surfaces, and the normals of the three second type point cloud surfaces, the second type normals of each of the three intersecting calibration surfaces are obtained.

[0015] According to preset weights, the first type normal and the second type normal of the same calibration surface are weighted to obtain the target normal of each of the three intersecting calibration surfaces.

[0016] In one embodiment, determining the first type of point cloud surface corresponding to each of the three intersecting calibration surfaces includes:

[0017] The point cloud data is segmented to obtain three first-class point cloud surfaces;

[0018] The normals of the three first-type point cloud surfaces in the coordinate system of the image acquisition device are used as the normals of the three first-type point cloud surfaces.

[0019] Based on the first type point cloud surface normal that is perpendicular to the other two first type point cloud surface normals among the three first type point cloud surface normals, and the relative size between the two included angles, the first type point cloud surface corresponding to each of the three intersecting calibration surfaces is determined among the three first type point cloud surfaces; the two included angles are the angles between the other two first type point cloud surface normals and the target coordinate axis of the image acquisition device coordinate system.

[0020] In one embodiment, based on the first-type point cloud surface normals that are perpendicular to the other two first-type point cloud surface normals among the three first-type point cloud surface normals, and the relative size between the two included angles, the first-type point cloud surface corresponding to each of the three intersecting calibration surfaces is determined from the three first-type point cloud surfaces, including:

[0021] Based on the first type of point cloud surface normal that is perpendicular to the other two first type of point cloud surface normals among the three first type of point cloud surface normals, determine the first type of point cloud surface corresponding to the perpendicular first type of point cloud surface normal among the three first type of point cloud surfaces.

[0022] Based on the relative size between the two included angles, the first type point cloud surface corresponding to the larger included angle and the first type point cloud surface corresponding to the smaller included angle are determined from the other two first type point cloud surfaces.

[0023] The first type of point cloud surface corresponding to the first calibration surface is determined to be the first type of point cloud surface corresponding to the normal of the first type of point cloud surface perpendicular to it; the first type of point cloud surface corresponding to the second calibration surface is the first type of point cloud surface corresponding to the large included angle; and the first type of point cloud surface corresponding to the third calibration surface is the first type of point cloud surface corresponding to the small included angle. Among the three intersecting calibration surfaces, the first calibration surface is perpendicular to the second calibration surface and the third calibration surface, and the angle between the second calibration surface and the target coordinate axis is greater than the angle between the third calibration surface and the target coordinate axis.

[0024] In one embodiment, determining the second type of point cloud surface corresponding to each of the three intersecting calibration surfaces includes:

[0025] The image acquisition device acquires the color image and depth image formed when it collects the point cloud data;

[0026] Based on the different colors of the three intersecting calibration surfaces, the regions where the three intersecting calibration surfaces are located are determined on the color image.

[0027] The regions where the three intersecting calibration surfaces are located are mapped to the depth image, and the second type of point cloud surface corresponding to each of the three intersecting calibration surfaces is determined in the point cloud data corresponding to the depth image.

[0028] In one embodiment, obtaining the transformation relationship between the image acquisition device coordinate system and the medical system coordinate system based on the three target normals and the normals of the three intersecting calibration surfaces in the medical system coordinate system includes:

[0029] Based on the three target normals and the normals of the three intersecting calibration surfaces in the medical system coordinate system, the rotation matrix between the image acquisition device coordinate system and the medical system coordinate system is obtained.

[0030] Based on the rotation matrix, the transformation relationship between the coordinate system of the image acquisition device and the coordinate system of the medical system is obtained.

[0031] In one embodiment, obtaining the transformation relationship between the image acquisition device coordinate system and the medical system coordinate system based on the rotation matrix includes:

[0032] The point cloud data is rotated using the rotation matrix, and the rotated point cloud data is then fitted.

[0033] Based on the fitting results, the position of the target corner point in the coordinate system of the image acquisition device is determined; the target corner point is the corner point of the calibration phantom acquired by the image acquisition device.

[0034] Based on the position of the target corner point in the image acquisition device coordinate system and the medical system coordinate system, the translation matrix between the image acquisition device coordinate system and the medical system coordinate system is obtained;

[0035] Based on the rotation matrix and the translation matrix, the transformation relationship between the coordinate system of the image acquisition device and the coordinate system of the medical system is obtained.

[0036] This application provides a coordinate system transformation device for a medical system, the device comprising:

[0037] The point cloud acquisition module is used to acquire point cloud data of the calibration phantom at a target location using an image acquisition device; the target location is the position of three intersecting calibration surfaces of the calibration phantom.

[0038] The segmentation and recognition module is used to segment the point cloud data to obtain three point cloud surfaces, and to determine the point cloud surface corresponding to each of the three intersecting calibration surfaces among the three point cloud surfaces;

[0039] The normal determination module is used to obtain three target normals based on the point cloud surfaces corresponding to the three intersecting calibration surfaces and the normals of the three point cloud surfaces in the coordinate system of the image acquisition device.

[0040] The transformation relationship acquisition module is used to obtain the transformation relationship between the image acquisition device coordinate system and the medical system coordinate system based on the three target normals and the normals of the three intersecting calibration surfaces in the medical system coordinate system.

[0041] This application provides a computer device, including a memory and a processor, wherein the memory stores a computer program and the processor executes the above-described method.

[0042] This application provides a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor using the methods described above.

[0043] This application provides a computer program product having a computer program stored thereon, the computer program being executed by a processor using the above-described method.

[0044] In the aforementioned medical system coordinate system transformation method, apparatus, computer equipment, storage medium, and computer program product, point cloud data of the calibration phantom is acquired by an image acquisition device at the target position of the three intersecting calibration surfaces of the calibration phantom; the point cloud data is segmented to obtain three point cloud surfaces; the point cloud surfaces corresponding to the three intersecting calibration surfaces are determined; based on the point cloud surfaces corresponding to the three intersecting calibration surfaces and the normals of the three point cloud surfaces in the coordinate system of the image acquisition device, the normals of the three intersecting calibration surfaces in the coordinate system of the image acquisition device are obtained; and combined with the normals of the three intersecting calibration surfaces in the medical system coordinate system, the transformation relationship between the image acquisition device coordinate system and the medical system coordinate system is obtained. Regardless of the location where the image acquisition device is installed, the point cloud segmentation and recognition technology provided in this application can be used to obtain the normals of the three intersecting calibration surfaces in the coordinate system of the image acquisition device, thereby obtaining the transformation relationship between the image acquisition device coordinate system and the medical system coordinate system, improving robustness. Attached Figure Description

[0045] Figure 1(a) shows a calibration scenario diagram in one embodiment;

[0046] Figure 1(b) is a schematic diagram of point cloud acquisition performed by the camera in one embodiment;

[0047] Figure 2 This is a flowchart illustrating a method for transforming the coordinate system of a medical system in one embodiment;

[0048] Figure 3 This is a schematic diagram of the calibration phantom in one embodiment;

[0049] Figure 4 This is a flowchart illustrating a method for transforming the coordinate system of a medical system in one embodiment;

[0050] Figure 5(a) is a flowchart illustrating the transformation method of the medical system coordinate system in one embodiment;

[0051] Figure 5(b) is a flowchart illustrating the transformation method of the medical system coordinate system in one embodiment;

[0052] Figure 6 This is a structural block diagram of a coordinate system transformation device for a medical system in one embodiment;

[0053] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0055] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0056] This application provides a method for transforming the coordinate system of a medical system. This method can be executed by a computer device. The following description is in conjunction with Figure 1(a), Figure 1(b), and... Figure 2 This method is described.

[0057] Step S201: At the target location, the point cloud data of the calibration phantom is acquired by an image acquisition device.

[0058] The tracking system includes a camera as the image acquisition device, and the coordinate system of the image acquisition device is called the camera coordinate system. The number of camera viewpoints included in the tracking system can be determined according to actual needs. In some scenarios, in order to acquire the left and right sides and the middle part of the abdomen of the radiotherapy subject, the tracking system can include three camera viewpoints. These three camera viewpoints are used to track the movement information of the left and right sides and the middle part of the abdomen of the radiotherapy subject, respectively. One camera can be installed in each camera viewpoint. In this case, the tracking system includes three cameras (camera, abbreviated as cam in the following description), as shown in cam1, cam2 and cam3 in Figure 1(a).

[0059] Each image acquisition device has its own coordinate system. For example, as shown in Figure 1(a), cam1 has its own coordinate system x1-y1-z1, cam2 has its own coordinate system x2-y2-z2, and cam3 has its own coordinate system x3-y3-z3.

[0060] The image acquisition device can acquire point cloud data of the calibration phantom, and the acquired point cloud data is in its own coordinate system. For example, as shown in Figure 1(b), cam1 acquires point cloud data of the calibration phantom, and the acquired point cloud data is located in the camera coordinate system x1-y1-z1 of cam1.

[0061] To perform calibration and obtain the transformation relationship between the image acquisition device coordinate system and the medical system coordinate system, a calibration phantom can be introduced. The pose of the calibration phantom in the image acquisition device coordinate system (hereinafter referred to as the first pose) and the pose of the calibration phantom in the medical system coordinate system (hereinafter referred to as the second pose) are obtained. The image acquisition device coordinate system or the medical system coordinate system is translated and rotated to match the first pose with the second pose. Based on the translation and rotation, the transformation relationship between the image acquisition device coordinate system and the medical system coordinate system is obtained to complete the calibration.

[0062] The calibration phantom can be a polyhedron; in one embodiment, the calibration phantom can be as follows: Figure 3 The decahedron shown.

[0063] When an image acquisition device is installed to collect motion information of a radiotherapy subject, during the calibration phase, the image acquisition device can capture three intersecting calibration surfaces of the calibration phantom. This location can be called the target location, that is, the target location is the position facing the three intersecting calibration surfaces of the calibration phantom.

[0064] In this step, when the image acquisition device is a depth camera, after the depth camera is installed at the target location, it is triggered to acquire point cloud data of the calibration phantom. Since the depth camera can face the three intersecting calibration surfaces of the calibration phantom at the target location, the acquired point cloud data corresponds to the three intersecting calibration surfaces. The point cloud data is acquired by the image acquisition device and is located in the coordinate system of the image acquisition device.

[0065] Step S202: Segment the point cloud data to obtain three point cloud surfaces. Among the three point cloud surfaces, determine the point cloud surface corresponding to each of the three intersecting calibration surfaces.

[0066] In this step, after obtaining the point cloud data, the point cloud data can be segmented and identified. Segmentation is mainly used to determine which points in the point cloud belong to the same surface, thus obtaining the point cloud surface. Identification is mainly used to determine which of the three intersecting calibration surfaces the point cloud surface corresponds to.

[0067] Step S203: Based on the point cloud surfaces corresponding to the three intersecting calibration surfaces and the normals of the three point cloud surfaces in the coordinate system of the image acquisition device, obtain the three target normals.

[0068] The target normal is the normal of the calibration surface in the coordinate system of the image acquisition device.

[0069] After segmenting and recognizing the point cloud data, the point cloud surfaces corresponding to the three intersecting calibration surfaces can be determined. Then, the normal of the point cloud surface in the coordinate system of the image acquisition device is taken as the normal of the calibration surface corresponding to that point cloud surface in the coordinate system of the image acquisition device. Thus, the normals of the three intersecting calibration surfaces in the coordinate system of the image acquisition device are obtained.

[0070] Step S204: Based on the normals of the three targets and the normals of the three intersecting calibration surfaces in the medical system coordinate system, the transformation relationship between the image acquisition device coordinate system and the medical system coordinate system is obtained.

[0071] The normals of the three intersecting calibration surfaces in the image acquisition device coordinate system can characterize the pose of the calibration phantom in the image acquisition device coordinate system (referred to as the first pose). Similarly, the normals of the three intersecting calibration surfaces in the medical system coordinate system can characterize the pose of the calibration phantom in the medical system coordinate system (referred to as the second pose).

[0072] Next, the coordinate system of the image acquisition device or the coordinate system of the medical system is translated and rotated to match the first pose with the second pose. Based on the translation and rotation, the transformation relationship between the coordinate system of the image acquisition device and the coordinate system of the medical system is obtained to complete the calibration.

[0073] In the above method, point cloud data of the calibration phantom is acquired by an image acquisition device at the target positions of the three intersecting calibration surfaces. The point cloud data is segmented to obtain three point cloud surfaces. Among the three point cloud surfaces, the point cloud surfaces corresponding to the three intersecting calibration surfaces are determined. Based on the point cloud surfaces corresponding to the three intersecting calibration surfaces and the normals of the three point cloud surfaces in the coordinate system of the image acquisition device, the normals of the three intersecting calibration surfaces in the coordinate system of the image acquisition device are obtained. Combined with the normals of the three intersecting calibration surfaces in the coordinate system of the medical system, the transformation relationship between the coordinate system of the image acquisition device and the coordinate system of the medical system is obtained. Regardless of the location where the image acquisition device is installed, the point cloud segmentation and recognition technology provided in this application can be used to obtain the normals of the three intersecting calibration surfaces in the coordinate system of the image acquisition device, thereby obtaining the transformation relationship between the coordinate system of the image acquisition device and the coordinate system of the medical system, improving robustness.

[0074] In one embodiment, the point cloud data is segmented to obtain three point cloud surfaces. Among the three point cloud surfaces, the point cloud surfaces corresponding to the three intersecting calibration surfaces are determined, including: segmenting and identifying the point cloud data, determining the first type of point cloud surface corresponding to each of the three intersecting calibration surfaces, and determining the second type of point cloud surface corresponding to each of the three intersecting calibration surfaces.

[0075] In this embodiment, the point cloud surface corresponding to the same calibration surface in different segmentation and recognition can be determined by multiple segmentation and recognition processes, thereby obtaining at least two normal lines of the same calibration surface in the coordinate system of the image acquisition device.

[0076] For example, in the first segmentation and recognition, the point cloud surface corresponding to a certain calibration surface can be determined, and the normal of the first determined point cloud surface in the image acquisition device is taken as the normal of the calibration surface in the image acquisition device; in the second segmentation and recognition, the point cloud surface corresponding to the calibration surface can be determined, and the normal of the second determined point cloud surface in the image acquisition device is taken as the normal of the calibration surface in the image acquisition device, thereby obtaining two normals of the calibration surface in the image acquisition device.

[0077] It is understandable that the point cloud surface determined in the second step may differ from the point cloud surface determined in the first step. The point cloud surfaces corresponding to the same calibration surface determined in different steps of segmentation and recognition can be referred to as the first type point cloud surface and the second type point cloud surface corresponding to that calibration surface, respectively. Each calibration surface can have its own corresponding first type point cloud surface and second type point cloud surface.

[0078] Since the point cloud surface determined in the second step may differ from the point cloud surface determined in the first step, the normals of these two point cloud surfaces in the image acquisition device may also differ. Therefore, the two normals of the calibration surface in the image acquisition device may be different. The two normals of the calibration surface in the coordinate system of the image acquisition device can be referred to as the first type normal and the second type normal of the calibration surface in the coordinate system of the image acquisition device, respectively.

[0079] It should be noted that the algorithms used for the different segmentation and recognition processes described above can be different.

[0080] In the above embodiments, the first type of point cloud surface and the second type of point cloud surface corresponding to the same calibration surface can be obtained through multiple point cloud segmentation and recognition. Then, based on at least two normals of the same calibration surface in the coordinate system of the image acquisition device, the normal of the same calibration surface in the coordinate system of the image acquisition device can be synthesized to improve the accuracy of the normal.

[0081] In one embodiment, three target normals are obtained based on the point cloud surfaces corresponding to the three intersecting calibration surfaces and the normals of the three point cloud surfaces in the coordinate system of the image acquisition device. This includes: obtaining the first-type normals of the three intersecting calibration surfaces based on the first-type point cloud surfaces corresponding to the three intersecting calibration surfaces and the normals of the three first-type point cloud surfaces; obtaining the second-type normals of the three intersecting calibration surfaces based on the second-type point cloud surfaces corresponding to the three intersecting calibration surfaces and the normals of the three second-type point cloud surfaces; and weighting the first-type normals and second-type normals of the same calibration surface according to a preset weight to obtain the target normals of the three intersecting calibration surfaces.

[0082] Among them, the normal of the first type of point cloud surface is the normal of the first type of point cloud surface in the coordinate system of the image acquisition device, and the normal of the second type of point cloud surface is the normal of the second type of point cloud surface in the coordinate system of the image acquisition device.

[0083] In this embodiment, when at least two normals of the same calibration surface in the coordinate system of the image acquisition device are obtained through multiple point cloud segmentation and recognition processes, the at least two normals can be weighted to obtain the target normal of the calibration surface in the coordinate system of the image acquisition device.

[0084] Specifically, the initial calculation involves segmenting and identifying the point cloud data, determining the point cloud surface corresponding to the first calibration surface as Point cloud surface_1_1, the point cloud surface corresponding to the second calibration surface as Point cloud surface_2_1, and the point cloud surface corresponding to the third calibration surface as Point cloud surface_3_1. The first, second, and third calibration surfaces are the three intersecting calibration surfaces mentioned above.

[0085] The point cloud surfaces corresponding to each calibration surface determined in this calculation can be called the first type of point cloud surface, that is, Pointcloud surface_1_1, Point cloud surface_2_1, and Point cloud surface_3_1 are called the first type of point cloud surface.

[0086] Next, the normal of Point cloud surface_1_1 in the image acquisition device coordinate system can be used as the normal of the first calibration surface in the image acquisition device coordinate system. In this way, the normals of the second calibration surface and the third calibration surface in the image acquisition device coordinate system can be obtained.

[0087] In this calculation, the normals of each calibration surface in the coordinate system of the image acquisition device can be called first-type normals, that is, the first-type normals of the intersecting calibration surfaces are obtained.

[0088] In the second calculation, after segmenting and identifying the point cloud data, the point cloud surface corresponding to the first calibration surface is determined to be Point cloud surface_1_2, the point cloud surface corresponding to the second calibration surface is determined to be Point cloud surface_2_2, and the point cloud surface corresponding to the third calibration surface is determined to be Point cloud surface_3_2.

[0089] The point cloud surfaces corresponding to each calibration surface determined in this calculation can be called second-type point cloud surfaces, that is, Pointcloud surface_1_2, Point cloud surface_2_2, and Point cloud surface_3_2 are called second-type point cloud surfaces.

[0090] Next, the normal of Point cloud surface_1_2 in the image acquisition device coordinate system can be used as the normal of the first calibration surface in the image acquisition device coordinate system. In this way, the normals of the second calibration surface and the third calibration surface in the image acquisition device coordinate system can be obtained.

[0091] In this calculation, the normals of each calibration surface in the coordinate system of the image acquisition device can be called second-type normals, that is, the second-type normals of the intersecting calibration surfaces are obtained.

[0092] After at least two calculations, the first and second type normals of the first calibration surface can be obtained. These normals are then weighted according to a preset weight, and the weighted result is used as the target normal of the first calibration surface. Similarly, the first and second type normals of the second calibration surface can be weighted in the same way, and the weighted result is used as the target normal of the second calibration surface. The same applies to the third calibration surface.

[0093] In this embodiment, when at least two normals of the same calibration surface in the coordinate system of the image acquisition device are obtained, the at least two normals can be weighted to avoid inaccurate normals due to a certain calculation deviation, thereby improving the accuracy of the pose of the calibration phantom in the coordinate system of the image acquisition device and improving the calibration accuracy.

[0094] In one embodiment, the first type of normal and the second type of normal of the same calibration surface are weighted according to a preset weight, including the following steps: calculating the included angle between two normals in the three normal pairs; determining whether the point cloud acquisition of the image acquisition device is successful based on the statistical results; if successful, the first type of normal and the second type of normal of the same calibration surface are weighted according to the preset weight.

[0095] Among them, the two normals included in the same normal pair are the first type normal and the second type normal of the same calibration plane.

[0096] After obtaining the first and second normals of the first calibration surface, the angle between the first and second normals of the first calibration surface can be calculated and denoted as ∠1; similarly, the angle between the first and second normals of the second calibration surface can be calculated and denoted as ∠2, and the angle between the first and second normals of the third calibration surface can be calculated and denoted as ∠3.

[0097] By averaging, ∠1, ∠2, and ∠3 are statistically analyzed. If the average result is less than a threshold, the point cloud acquisition by the image acquisition device is considered successful. At this point, the first and second normals of the first calibration surface can be weighted to obtain the target normal of the first calibration surface. Similarly, the first and second normals of the second calibration surface can be weighted to obtain the target normal of the second calibration surface. The first and second normals of the third calibration surface can be weighted to obtain the target normal of the third calibration surface.

[0098] In the above embodiments, the angle between the first type of normal and the second type of normal of each calibration surface is statistically analyzed and cross-validated. If the validation is successful, it means that the point cloud acquisition is successful and calibration can be performed to calibrate the accuracy.

[0099] In one embodiment, determining the first type of point cloud surface corresponding to each of the three intersecting calibration surfaces includes: segmenting the point cloud data to obtain three first type of point cloud surfaces; using the normals of the three first type of point cloud surfaces in the coordinate system of the image acquisition device as the normals of the three first type of point cloud surfaces; determining the first type of point cloud surface corresponding to each of the three intersecting calibration surfaces among the three first type of point cloud surfaces based on the normal of the first type of point cloud surface that is perpendicular to the normals of the other two first type of point cloud surfaces, and the relative size between the two included angles; the two included angles are the angles between the normals of the other two first type of point cloud surfaces and the target coordinate axes of the coordinate system of the image acquisition device.

[0100] In the point cloud segmentation and recognition algorithm used in this embodiment, segmentation and recognition are performed sequentially. That is, the point cloud data is first segmented to obtain point cloud surfaces, and then the point cloud surfaces are recognized to determine which calibration surface the point cloud surface corresponds to.

[0101] Specifically, spatial plane fitting technology can be used to segment point cloud data to obtain three first-class point cloud surfaces, and then the normals of the three first-class point cloud surfaces in the coordinate system of the image acquisition device can be obtained. These normals are called first-class point cloud surface normals.

[0102] Among the three type I point cloud surface normals, one type I point cloud surface normal is perpendicular to the other two type I point cloud surface normals. The other two type I point cloud surface normals each have a certain relative size relationship with the target coordinate axis of the image acquisition device coordinate system. The target coordinate axis can be the x-axis of the image acquisition device coordinate system.

[0103] Based on the normals of the first type point cloud surfaces that are perpendicular to the normals of the other two first type point cloud surfaces, and the relative size between the two included angles, the first type point cloud surfaces corresponding to the three intersecting calibration surfaces are determined among the three first type point cloud surfaces.

[0104] In the above embodiments, the point cloud data is first segmented, and then based on the spatial positional relationship between the normals, the point cloud surfaces corresponding to each calibration surface are identified according to the perpendicular relationship between the normals of the first type of point cloud surface and the relative size of the angle between the normals of the first type of point cloud surface and the target coordinate axis. This eliminates the need to introduce too many other factors to complete the identification, thus improving the identification efficiency.

[0105] In one embodiment, among the three intersecting calibration planes, the first calibration plane is perpendicular to the second and third calibration planes, and the angle between the second calibration plane and the target coordinate axis is greater than the angle between the third calibration plane and the target coordinate axis. This information can be considered as prior knowledge.

[0106] In this case, based on the first-type point cloud surface normals that are perpendicular to the other two first-type point cloud surface normals among the three first-type point cloud surface normals, and the relative size between the two included angles, the first-type point cloud surface corresponding to each of the three intersecting calibration surfaces can be determined from the three first-type point cloud surfaces. This can specifically include the following steps:

[0107] Based on the first-type point cloud surface normals that are perpendicular to the normals of the other two first-type point cloud surfaces among the three first-type point cloud surfaces, the first-type point cloud surface corresponding to the perpendicular first-type point cloud surface normal is determined among the three first-type point cloud surfaces; according to the relative size between the two included angles, the first-type point cloud surface corresponding to the larger included angle and the first-type point cloud surface corresponding to the smaller included angle are determined among the other two first-type point cloud surfaces; the first-type point cloud surface corresponding to the first calibration surface is determined to be the first-type point cloud surface corresponding to the perpendicular first-type point cloud surface normal, the first-type point cloud surface corresponding to the second calibration surface is the first-type point cloud surface corresponding to the larger included angle, and the first-type point cloud surface corresponding to the third calibration surface is the first-type point cloud surface corresponding to the smaller included angle.

[0108] During the identification phase, this embodiment combines prior knowledge and the spatial relationship between normals to determine the point cloud surfaces corresponding to the first calibration surface, the second calibration surface, and the third calibration surface.

[0109] Specifically, the first type of point cloud surfaces obtained after segmenting the point cloud data are: Point cloud surface_1_1, Point cloud surface_2_1, and Point cloud surface_3_1. In these three first-type point cloud surfaces, the normal of Point cloud surface_1_1 in the image acquisition device coordinate system is perpendicular not only to the normal of Point cloud surface_2_1 in the image acquisition device coordinate system, but also to the normal of Point cloud surface_3_1 in the image acquisition device coordinate system.

[0110] Based on the prior knowledge mentioned above, among the three intersecting calibration surfaces, the first calibration surface is perpendicular to the second calibration surface and also perpendicular to the third calibration surface. Therefore, Point cloud surface_1_1 can be associated with the first calibration surface, that is, the first type of point cloud surface corresponding to the first calibration surface is Point cloud surface_1_1.

[0111] For the two first-type point cloud surfaces, Point cloud surface_2_1 and Point cloud surface_3_1, we can obtain the angle between the normal of Point cloud surface_2_1 in the image acquisition device coordinate system and the x-axis of the image acquisition device coordinate system, and the angle between the normal of Point cloud surface_3_1 in the image acquisition device coordinate system and the x-axis of the image acquisition device coordinate system. After obtaining these two angles, we can determine the relative size relationship between them.

[0112] Based on the prior knowledge mentioned above, we know that the angle between the second calibration surface and the target coordinate axis is greater than the angle between the third calibration surface and the target coordinate axis. Therefore, if the point cloud surface corresponding to the larger angle is Point cloud surface_2_1, then according to the prior knowledge mentioned above, Point cloud surface_2_1 can be associated with the second calibration surface, and the point cloud surface Point cloud surface_3_1 corresponding to the smaller angle can be associated with the third calibration surface.

[0113] In the above embodiments, based on prior knowledge related to the calibration phantom, the spatial positional relationship between the normals of the three first-type point cloud surfaces, and the spatial positional relationship between the normals of the first-type point cloud surfaces, the calibration surface corresponding to each point cloud surface can be identified more accurately, and there is no need to introduce a complex neural network model, thus improving the recognition efficiency.

[0114] In one embodiment, determining the second type of point cloud surface corresponding to each of the three intersecting calibration surfaces includes: acquiring a color image and a depth image formed when the image acquisition device collects point cloud data of the calibration phantom; determining the region where each of the three intersecting calibration surfaces is located on the color image based on the different colors of the three intersecting calibration surfaces; mapping the region where each of the three intersecting calibration surfaces is located to the depth image, and determining the second type of point cloud surface corresponding to each of the three intersecting calibration surfaces in the point cloud data corresponding to the depth image.

[0115] In the point cloud segmentation and recognition algorithm used in this embodiment, there is no strict distinction between the order of segmentation and recognition. That is, by using the point cloud segmentation and recognition algorithm used in this embodiment, while obtaining the point cloud surface, it is also possible to determine which calibration surface the point cloud surface corresponds to.

[0116] Specifically, the three intersecting calibration surfaces mentioned above each have a different color, which can be blue, green, and red, respectively. The image acquisition device can be an RGB-D camera (R is short for red; G is short for green; B is short for blue; D is short for depth). When the point cloud data of the calibration phantom is acquired using an RGB-D camera, corresponding color images and depth images will be obtained.

[0117] Next, based on the different colors of the three intersecting calibration surfaces, the regions where the first calibration surface, the second calibration surface, and the third calibration surface are located are determined on the color image.

[0118] Since point cloud data can be obtained by converting depth images, there is a correspondence between depth images and point cloud data. After mapping the area where the first calibration surface is located to the depth image, the second type of point cloud surface corresponding to the first calibration surface can be determined in the point cloud data corresponding to the depth image. In this way, the second type of point cloud surface corresponding to the second calibration surface and the second type of point cloud surface corresponding to the third calibration surface can be determined.

[0119] In the above embodiments, based on the color features of the calibration surface itself, the point cloud data is segmented and identified using color images and depth images. Even when prior knowledge such as the spatial relationship between calibration surfaces is unavailable, segmentation and identification can still be effectively completed.

[0120] In one embodiment, the transformation relationship between the image acquisition device coordinate system and the medical system coordinate system is obtained based on the three target normals and the normals of the three intersecting calibration surfaces in the medical system coordinate system. This includes: obtaining the rotation matrix between the image acquisition device coordinate system and the medical system coordinate system based on the three target normals and the normals of the three intersecting calibration surfaces in the medical system coordinate system; and obtaining the transformation relationship between the image acquisition device coordinate system and the medical system coordinate system based on the rotation matrix.

[0121] The three target normals characterize the pose of the calibration phantom in the image acquisition device coordinate system, and the normals of the three intersecting calibration surfaces in the medical system coordinate system characterize the pose of the calibration phantom in the medical system coordinate system. Based on the three target normals and the normals of the three intersecting calibration surfaces in the medical system coordinate system, the rotation matrix between the image acquisition device coordinate system and the medical system coordinate system is obtained. Based on the rotation matrix, the transformation relationship between the image acquisition device coordinate system and the medical system coordinate system is obtained.

[0122] In the above embodiments, the transformation relationship between the coordinate system of the image acquisition device and the coordinate system of the medical system is obtained from the rotation angle. In subsequent optical surface radiotherapy tracking, the motion information of the radiotherapy object can be accurately transmitted to the medical system.

[0123] In one embodiment, the transformation relationship between the image acquisition device coordinate system and the medical system coordinate system is obtained based on the rotation matrix, specifically including the following steps: rotating the point cloud data using the rotation matrix and fitting the rotated point cloud data; determining the position of the target corner point in the image acquisition device coordinate system based on the fitting result; obtaining the translation matrix between the image acquisition device coordinate system and the medical system coordinate system based on the position of the target corner point in the image acquisition device coordinate system and the medical system coordinate system; and obtaining the transformation relationship between the image acquisition device coordinate system and the medical system coordinate system based on the rotation matrix and the translation matrix.

[0124] The corner points of a calibration phantom are the points formed by the intersection of calibration surfaces. When the image acquisition device performs point cloud acquisition on the three intersecting calibration surfaces of the calibration phantom, the corner points corresponding to the three intersecting calibration surfaces will also be acquired by the image acquisition device, and these corner points can be called target corner points.

[0125] After obtaining the rotation matrix, the point cloud data can be rotated, and the rotated point cloud data can be fitted. Based on the fitting result, the position of the target corner point in the coordinate system of the image acquisition device can be determined. Combined with the position of the target corner point in the coordinate system of the medical system, the translation matrix between the coordinate system of the image acquisition device and the coordinate system of the medical system can be obtained.

[0126] Based on the rotation and translation matrices described above, the transformation relationship between the image acquisition device coordinate system and the medical system coordinate system is obtained.

[0127] In the above embodiments, the transformation relationship between the coordinate system of the image acquisition device and the coordinate system of the medical system is obtained from the two angles of rotation and translation. In subsequent optical surface radiotherapy tracking, the motion information of the radiotherapy object can be accurately transmitted to the medical system. Furthermore, the position of the corner point of the calibration phantom in the coordinate system of the image acquisition device and the coordinate system of the medical system is relatively accurate, thereby obtaining a relatively accurate translation matrix.

[0128] To better understand the above method, the following is a detailed application example of the coordinate system transformation method of the medical system in this application.

[0129] In this application example, the medical system is a radiotherapy system. The tracking system includes a camera as the image acquisition device, and the coordinate system of the image acquisition device is called the camera coordinate system. The number of camera viewpoints included in the tracking system can be determined according to actual needs. In some scenarios, in order to acquire the left and right sides and the middle part of the abdomen of the radiotherapy subject, the tracking system can include three camera viewpoints. These three camera viewpoints are used to track the movement information of the left and right sides and the middle part of the abdomen of the radiotherapy subject, respectively. One camera can be installed in each camera viewpoint. In this case, the tracking system includes three cameras, as shown in Figure 1(a), cam1, cam2 and cam3.

[0130] Each camera has its own coordinate system. For example, as shown in Figure 1(a), cam1 has its own coordinate system x1-y1-z1, cam2 has its own coordinate system x2-y2-z2, and cam3 has its own coordinate system x3-y3-z3.

[0131] The camera can acquire point cloud data of the calibration phantom, and the acquired point cloud data is in its own coordinate system. For example, as shown in Figure 1(b), cam1 acquires point cloud data of the calibration phantom, and the acquired point cloud data is located in the camera coordinate system x1-y1-z1 of cam1.

[0132] Three intersecting surfaces of a calibration phantom under the same camera view are considered as a group. The calibration phantom provided in this application example includes multiple groups of calibration surfaces, indicating that the calibration phantom has three intersecting surfaces under different camera viewpoints.

[0133] In the same group of calibration surfaces, the first calibration surface is perpendicular to the second and third calibration surfaces respectively; the normal of the second calibration surface in the coordinate system of the image acquisition device forms a first angle with the target coordinate axis of the coordinate system of the image acquisition device; the normal of the third calibration surface in the coordinate system of the image acquisition device forms a second angle with the target coordinate axis; the size relationship between the first angle and the second angle is consistent in each group of calibration surfaces.

[0134] Taking the calibration phantom shown in Figure 1(a) as an example, the upper surface of the calibration phantom is denoted as i, and the side surfaces are denoted as a, b, c, d, e, f, and g, respectively. From the perspective of camera cam1, if cam1 can see three intersecting surfaces a, b, and i, then these three intersecting surfaces are considered a group. From the perspective of camera cam2, if cam2 can see three intersecting surfaces g, f, and i, then these three intersecting surfaces are considered a group. From the perspective of camera cam3, if cam3 can see three intersecting surfaces e, d, and i, then these three intersecting surfaces are considered a group. Therefore, the calibration phantom is said to include multiple groups of calibration surfaces. The number of groups can be determined according to the number of camera perspectives included in the tracking system. For example, when there are 3 camera perspectives, the calibration phantom includes 3 groups of calibration surfaces, with each camera perspective having its corresponding group.

[0135] Within the same group, one calibration plane is perpendicular to the other two calibration planes; this calibration plane is called the first calibration plane. Of the remaining two calibration planes, the one located to the left of the camera's line of sight is called the second calibration plane, and the one located to the right of the camera's line of sight is called the third calibration plane. For example, in the group of calibration planes {a, b, i}, i is perpendicular to a, i is perpendicular to b, and i is called the first calibration plane. Of the remaining two planes a and b, a is located to the left of cam1's line of sight, and a is called the second calibration plane; b is located to the right of cam1's line of sight, and b is called the third calibration plane.

[0136] The first calibration surface can be the upper surface of the calibration phantom, and the second and third calibration surfaces can be the adjacent side surfaces of the calibration phantom. The side surfaces of the calibration phantom can be rectangular. The size of the second calibration surface can be larger than the size of the third calibration surface.

[0137] In each set of calibration surfaces included in the calibration phantom, one calibration surface is perpendicular to the other two calibration surfaces. For example, in the three sets of calibration surfaces {a, b, i}, {g, f, i}, and {e, d, i}, there is a surface i that is perpendicular to the other two surfaces in the same set.

[0138] The angle formed by the normal of the second calibration surface in the camera coordinate system and the target coordinate axis in the camera coordinate system is called the first angle; the angle formed by the normal of the third calibration surface in the camera coordinate system and the target coordinate axis is called the second angle.

[0139] Taking a set of calibration surfaces {a, b, i} from the perspective of the cam1 camera as an example, if a is the second calibration surface, b is the third calibration surface, and the x1 axis of the cam1 camera coordinate system is the target coordinate axis, then the angle ∠a-x1 between a and x1 belongs to the first angle, and the angle ∠b-x1 between b and x1 belongs to the second angle.

[0140] Similarly, in a set of calibration surfaces {g, f, i} from the perspective of the cam2 camera, in this example, if g is the second calibration surface, f is the third calibration surface, and the x2 axis of the cam2 camera coordinate system is the target coordinate axis, then the angle ∠g-x2 between g and x2 belongs to the first angle, and the angle ∠f-x2 between f and x2 belongs to the second angle.

[0141] Similarly, in a set of calibration surfaces {e, d, i} from the perspective of the cam3 camera, in this example, if e is the second calibration surface, d is the third calibration surface, and the x3 axis of the cam3 camera coordinate system is the target coordinate axis, then the angle ∠e-x3 between e and x3 belongs to the first angle, and the angle ∠d-x3 between d and x3 belongs to the second angle.

[0142] The relationship between the first included angle and the second included angle is consistent across all calibration surfaces.

[0143] Taking the above three sets of calibration surfaces as examples, the size relationships between ∠a-x1 and ∠b-x1, ∠g-x2 and ∠f-x2, and ∠e-x3 and ∠d-x3 are kept consistent. That is, ∠a-x1 is greater than ∠b-x1, ∠g-x2 is greater than ∠f-x2, and ∠e-x3 is greater than ∠d-x3. At this time, the angle between the second calibration surface and the target coordinate axis is greater than the angle between the third calibration surface and the target coordinate axis.

[0144] Among them, the first calibration surface, the second calibration surface, and the third calibration surface in the same group have different colors.

[0145] For example, in {a, b, i}, a is green, b is blue, and i is red; and in another example, in {g, f, i}, g is yellow, f is purple, and i is black.

[0146] Among them, the first calibration surfaces of different groups have the same color, the second calibration surfaces of different groups have the same color, and the third calibration surfaces of different groups have the same color.

[0147] For example, in {a, b, i}, {g, f, i} and {e, d, i}, i is the first calibration surface, and the color of each group of i is the same, which can be set to red. a, g and e belong to the second calibration surface and can all be set to green. b, f and d belong to the third calibration surface and can all be set to blue.

[0148] Among them, the surface size of the calibration phantom is positively correlated with the target distance; the target distance is the distance between the installation position of the image acquisition device and the placement position of the calibration phantom.

[0149] The surface dimensions of the calibration phantom can be determined based on the dimensions of each facet of the phantom. To improve calibration accuracy, the surface dimensions of the calibration phantom can be adaptively increased as the image acquisition device moves further away from the phantom.

[0150] The calibration mold has three laser bonding slots; the laser bonding slots are used to bond the laser to position the calibration mold; the laser emission directions corresponding to the three laser bonding slots are perpendicular to each other.

[0151] When positioning the calibration phantom, a laser and laser bonding slots on the phantom are required. The laser can be a surface laser. Since the positioning of the calibration phantom is three-dimensional, the laser is emitted from three directions: x, y, and z, and these three laser emission directions are perpendicular to each other. Accordingly, there are three laser bonding slots on the calibration phantom: one for bonding the laser emitted from the x direction, one for bonding the laser emitted from the y direction, and one for bonding the laser emitted from the z direction.

[0152] When the three laser bonding slots can be aligned with their respective lasers, the setup is successful, and point cloud acquisition can then be performed.

[0153] Figure 3 The calibration phantom shown includes ten surfaces: an upper surface i, a lower surface, and eight side surfaces. The upper surface i corresponds to the first calibration surface, and both the upper and lower surfaces are octagonal. The eight side surfaces are composed of small blue rectangular surfaces and large green rectangular surfaces arranged sequentially, and each surface has three laser bonding grooves.

[0154] This application example uses the aforementioned calibration phantom to obtain the transformation relationship between the camera coordinate system and the radiotherapy system coordinate system. The following example uses the transformation relationship between the camera coordinate system of cam1 and the radiotherapy system coordinate system as an example to illustrate the specific calibration process. Figure 4 The steps shown are as follows:

[0155] Step S401: After cam1 is aligned with the three intersecting calibration surfaces of the calibration model, point cloud data of the calibration model is acquired through cam1; the point cloud data is in the camera coordinate system of cam1.

[0156] Step S402: When cam1 is an RGB-D camera, acquire the color image and depth image formed when cam1 acquires the point cloud data of the calibration model;

[0157] Step S403: Using one or two methods, segment and identify the point cloud data to determine the point cloud surface corresponding to each of the three intersecting calibration surfaces;

[0158] Step S404: Based on the normals of the point cloud surface in the camera coordinate system of cam1, obtain the normals of the three intersecting calibration surfaces in the camera coordinate system of cam1.

[0159] Step S405: Based on the normals of the three intersecting calibration surfaces in the camera coordinate system of cam1 and the normals in the radiotherapy system coordinate system, obtain the transformation relationship between the camera coordinate system of cam1 and the radiotherapy system coordinate system, and complete the coarse registration.

[0160] Step S406: Use the point cloud ICP (Iterative Closest Point) registration algorithm to complete the fine registration.

[0161] The following describes two available methods involved in step S403:

[0162] Available Method 1: Point cloud segmentation and recognition have a sequential order.

[0163] As shown in Figure 5(a), after cam1 is installed at the target position, cam1 is positioned opposite calibration surfaces a, b, and i of the calibration model. Calibration surfaces a and b are side surfaces, calibration surface i is the top surface, and calibration surface i is perpendicular to calibration surfaces a and b. The angle between calibration surface a and the x-axis of the camera coordinate system of cam1 is greater than the angle between calibration surface b and the x-axis of the camera coordinate system of cam1.

[0164] In the point cloud acquisition stage, after acquiring the point cloud data of the calibration phantom through cam1, point cloud data located in the camera coordinate system of cam1 can be obtained. Next, in the point cloud segmentation stage, spatial plane fitting technology is used to segment the point cloud data to obtain three point cloud surfaces, which are also located in the camera coordinate system of cam1. Then, in the point cloud recognition stage, the three point cloud surfaces are identified to determine which of the calibration surfaces a, b, and i they correspond to.

[0165] In the recognition phase, specifically, the normals of the three point cloud surfaces in the camera coordinate system of cam1 can be obtained. Then, the angles between each pair of normals are calculated to determine that one normal is perpendicular to the other two. Combining this with prior knowledge of the calibration phantom, it can be determined that the point cloud surface corresponding to this normal corresponds to calibration surface i. Next, the angles between the other two normals and the x-axis can be calculated. Among these two normals, the normal with the larger angle to the x-axis is determined, and the point cloud surface corresponding to this normal corresponds to calibration surface a. The normal with the smaller angle to the x-axis is determined, and the point cloud surface corresponding to this normal corresponds to calibration surface b.

[0166] Method 2: There is no strict order between point cloud segmentation and recognition.

[0167] As shown in Figure 5(b), after cam1 is installed at the target position, cam1 is aligned with calibration surfaces a, b, and i of the calibration phantom. Calibration surfaces a, b, and i are green, blue, and red, respectively.

[0168] During the point cloud acquisition stage, point cloud data of the calibration phantom in the camera coordinate system of cam1 can be obtained through point cloud acquisition using cam1, and corresponding color and depth images can be obtained. Next, the point cloud segmentation and recognition stage is entered. Based on the colors in the color image, it can be determined that the green area is the area where calibration surface a is located, the blue area is the area where calibration surface b is located, and the red area is the area where calibration surface i is located. Then, the green, blue, and red areas are mapped to the depth image. In the point cloud data corresponding to the depth image, the point cloud of the green area is determined to obtain the corresponding point cloud surface, which corresponds to calibration surface a. Similarly, the point cloud of the blue area is determined to obtain the corresponding point cloud surface, which corresponds to calibration surface b. The point cloud of the red area is determined to obtain the corresponding point cloud surface, which corresponds to calibration surface i.

[0169] If step S403 uses two methods for segmentation and recognition, then two normals of the same calibration surface in the camera coordinate system of cam1 can be obtained. At this time, these two normals can be weighted and the weighted normals can be used for calibration.

[0170] It is understandable that the transformation relationship between the camera coordinate system and the radiotherapy system coordinate system of cam2 and cam3 can also be obtained in the above way.

[0171] This application example fully considers the integrity and complete automation of the calibration algorithm, providing both coarse and fine registration functions. The coarse and fine point cloud registrations are seamlessly integrated, automatically completing the entire calibration process without any manual intervention. Furthermore, this coarse registration algorithm utilizes only the common spatial geometric features of the calibration phantom, eliminating the need for prior information during installation and placement, or the design of complex geometric features for coarse registration. This ensures reliability and stability, preventing the introduction of errors and randomness. Additionally, this coarse registration algorithm exhibits high robustness, utilizing both the spatial geometric and color information of the calibration phantom. If one type of information fails, the other remains usable, guaranteeing algorithm stability. Moreover, this coarse registration algorithm boasts high accuracy. It simultaneously uses both spatial geometric and color information from the calibration phantom. If one type of information is inaccurate, the other can be cross-validated, fully leveraging both types of information to obtain more precise coarse registration data. This improves both the coarse registration accuracy and the overall accuracy of the registration algorithm. This registration algorithm has a very high degree of adaptability and can be directly used for calibration of multiple cameras installed from different angles. The operation is uniform and easy to use.

[0172] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0173] In one embodiment, such as Figure 6 As shown, a medical system coordinate system transformation device is provided, comprising:

[0174] The point cloud acquisition module 601 is used to acquire point cloud data of the calibration phantom at a target location using an image acquisition device; the target location is the position of three intersecting calibration surfaces of the calibration phantom.

[0175] The segmentation and recognition module 602 is used to segment the point cloud data to obtain three point cloud surfaces, and to determine the point cloud surface corresponding to each of the three intersecting calibration surfaces among the three point cloud surfaces;

[0176] The normal determination module 603 is used to obtain three target normals based on the point cloud surfaces corresponding to the three intersecting calibration surfaces and the normals of the three point cloud surfaces in the coordinate system of the image acquisition device; the target normals are the normals of the calibration surfaces in the coordinate system of the image acquisition device.

[0177] The transformation relationship acquisition module 604 is used to obtain the transformation relationship between the image acquisition device coordinate system and the medical system coordinate system based on the three target normals and the normals of the three intersecting calibration surfaces in the medical system coordinate system.

[0178] In one embodiment, the segmentation and recognition module 602 is further configured to segment and recognize the point cloud data, determine the first type of point cloud surface corresponding to each of the three intersecting calibration surfaces, and determine the second type of point cloud surface corresponding to each of the three intersecting calibration surfaces.

[0179] In one embodiment, the normal determination module 603 is used to obtain the first type normal of each of the three intersecting calibration surfaces based on the first type point cloud surface corresponding to each of the three intersecting calibration surfaces and the normals of the three first type point cloud surfaces; to obtain the second type normal of each of the three intersecting calibration surfaces based on the second type point cloud surface corresponding to each of the three intersecting calibration surfaces and the normals of the three second type point cloud surfaces; and to weight the first type normal and the second type normal of the same calibration surface according to a preset weight to obtain the target normal of each of the three intersecting calibration surfaces.

[0180] In one embodiment, the segmentation and recognition module 602 is further configured to segment the point cloud data to obtain three first-type point cloud surfaces; take the normals of the three first-type point cloud surfaces in the coordinate system of the image acquisition device as the normals of the three first-type point cloud surfaces; based on the normals of the first-type point cloud surfaces that are perpendicular to the normals of the other two first-type point cloud surfaces, and the relative size between the two included angles, determine the first-type point cloud surfaces corresponding to the three intersecting calibration surfaces among the three first-type point cloud surfaces; the two included angles are the angles between the normals of the other two first-type point cloud surfaces and the target coordinate axes of the coordinate system of the image acquisition device.

[0181] In one embodiment, the segmentation and recognition module 602 is further configured to: determine the first type of point cloud surface corresponding to the perpendicular first type of point cloud surface normal among the three first type of point cloud surface normals; determine the first type of point cloud surface corresponding to the larger angle and the first type of point cloud surface corresponding to the smaller angle among the other two first type of point cloud surfaces based on the relative size between the two included angles; determine the first type of point cloud surface corresponding to the first calibration surface as the first type of point cloud surface corresponding to the perpendicular first type of point cloud surface normal, the first type of point cloud surface corresponding to the second calibration surface as the first type of point cloud surface corresponding to the larger angle, and the first type of point cloud surface corresponding to the third calibration surface as the first type of point cloud surface corresponding to the smaller angle; among the three intersecting calibration surfaces, the first calibration surface is perpendicular to the second calibration surface and the third calibration surface, and the angle between the second calibration surface and the target coordinate axis is greater than the angle between the third calibration surface and the target coordinate axis.

[0182] In one embodiment, the segmentation and recognition module 602 is further configured to acquire the color image and depth image formed when the image acquisition device collects point cloud data; determine the region where each of the three intersecting calibration surfaces is located on the color image based on the different colors of the three intersecting calibration surfaces; map the region where each of the three intersecting calibration surfaces is located to the depth image; and determine the second type of point cloud surface corresponding to each of the three intersecting calibration surfaces in the point cloud data corresponding to the depth image.

[0183] In one embodiment, the transformation relationship acquisition module 604 is used to obtain the rotation matrix between the image acquisition device coordinate system and the medical system coordinate system based on the three target normals and the normals of the three intersecting calibration surfaces in the medical system coordinate system; and to obtain the transformation relationship between the image acquisition device coordinate system and the medical system coordinate system based on the rotation matrix.

[0184] In one embodiment, the transformation relationship acquisition module 604 is further configured to: rotate the point cloud data using a rotation matrix; fit the rotated point cloud data; determine the position of the target corner point in the coordinate system of the image acquisition device based on the fitting result; the target corner point is the corner point of the calibration phantom acquired by the image acquisition device; obtain the translation matrix between the image acquisition device coordinate system and the medical system coordinate system based on the position of the target corner point in the coordinate system of the image acquisition device and the medical system coordinate system; and obtain the transformation relationship between the image acquisition device coordinate system and the medical system coordinate system based on the rotation matrix and the translation matrix.

[0185] Specific limitations regarding the medical system coordinate system transformation device can be found in the limitations of the medical system coordinate system transformation method described above, and will not be repeated here. Each module in the aforementioned medical system coordinate system transformation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of the computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0186] In one embodiment, a computer device is provided, the internal structure of which can be shown as follows: Figure 7 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores transformation data for the medical system coordinate system. The network interface communicates with external terminals via a network connection. The computer device also includes input / output interfaces, which are connection circuits between the processor and external devices for exchanging information; they are connected to the processor via a bus and are referred to as I / O interfaces. When the computer program is executed by the processor, it implements a method for transforming the medical system coordinate system.

[0187] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0188] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps in the various method embodiments described above.

[0189] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the various method embodiments described above.

[0190] In one embodiment, a computer program product is provided having a computer program stored thereon, the computer program being executed by a processor of the steps described in the various method embodiments above.

[0191] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0192] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0193] The above embodiments are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for transforming a coordinate system in a medical system, characterized in that, The method includes: At the target location, point cloud data of the calibration phantom is acquired using an image acquisition device; the target location is the position of three intersecting calibration surfaces of the calibration phantom. The point cloud data is segmented to obtain three point cloud surfaces. Among the three point cloud surfaces, the point cloud surface corresponding to each of the three intersecting calibration surfaces is determined. Based on the point cloud surfaces corresponding to the three intersecting calibration surfaces, and the normals of the three point cloud surfaces in the coordinate system of the image acquisition device, three target normals are obtained. Based on the three target normals and the normals of the three intersecting calibration surfaces in the medical system coordinate system, the transformation relationship between the image acquisition device coordinate system and the medical system coordinate system is obtained. The three intersecting calibration surfaces have geometric and / or color features; The geometric features include: the first calibration plane of the three intersecting calibration planes is perpendicular to the second and third calibration planes, and the angle between the second calibration plane and the target coordinate axis is greater than the angle between the third calibration plane and the target coordinate axis; The color feature includes: three intersecting calibration surfaces each having a different color; The point cloud data is segmented to obtain three point cloud surfaces. Among these three point cloud surfaces, the point cloud surfaces corresponding to the three intersecting calibration surfaces are determined. Based on the point cloud surfaces corresponding to the three intersecting calibration surfaces and the normals of the three point cloud surfaces in the image acquisition device coordinate system, three target normals are obtained, including: Based on the geometric features, the point cloud data is segmented and identified to determine the first type of point cloud surface corresponding to each of the three intersecting calibration surfaces. Based on the normals of the first type of point cloud surfaces corresponding to each of the three intersecting calibration surfaces in the coordinate system of the image acquisition device, the target normals of each of the three intersecting calibration surfaces are obtained. Alternatively, based on the color features, the point cloud data is segmented and identified to determine the second type of point cloud surface corresponding to each of the three intersecting calibration surfaces. Based on the normals of the second type of point cloud surfaces corresponding to each of the three intersecting calibration surfaces in the coordinate system of the image acquisition device, the target normals of each of the three intersecting calibration surfaces are obtained. Alternatively, based on the geometric features, the point cloud data is segmented and identified to determine the first type of point cloud surface corresponding to each of the three intersecting calibration surfaces. Based on the normals of the first type of point cloud surfaces corresponding to each of the three intersecting calibration surfaces in the image acquisition device coordinate system, the first type of normals of each of the three intersecting calibration surfaces are obtained. Based on the color features, the point cloud data is segmented and identified to determine the second type of point cloud surface corresponding to each of the three intersecting calibration surfaces. Based on the normals of the second type of point cloud surfaces corresponding to each of the three intersecting calibration surfaces in the image acquisition device coordinate system, the second type of normals of each of the three intersecting calibration surfaces are obtained. The first type of normals and the second type of normals of the same calibration surface are weighted to obtain the target normals of each of the three intersecting calibration surfaces.

2. The method according to claim 1, characterized in that, Weighting of the first-type and second-type normals of the same calibration surface includes: Based on preset weights, the first-type normals and the second-type normals of the same calibration surface are weighted.

3. The method according to claim 1, characterized in that, Based on the geometric features, the point cloud data is segmented and identified to determine the first type of point cloud surface corresponding to each of the three intersecting calibration surfaces, including: The point cloud data is segmented to obtain three first-class point cloud surfaces; The normals of the three first-type point cloud surfaces in the coordinate system of the image acquisition device are used as the normals of the three first-type point cloud surfaces. Based on the first type point cloud surface normal that is perpendicular to the other two first type point cloud surface normals among the three first type point cloud surface normals, and the relative size between the two included angles, the first type point cloud surface corresponding to each of the three intersecting calibration surfaces is determined among the three first type point cloud surfaces; the two included angles are the angles between the other two first type point cloud surface normals and the target coordinate axis of the image acquisition device coordinate system.

4. The method according to claim 3, characterized in that, Based on the first-type point cloud surface normals that are perpendicular to the other two first-type point cloud surface normals among the three first-type point cloud surface normals, and the relative size between the two included angles, the first-type point cloud surface corresponding to each of the three intersecting calibration surfaces is determined from the three first-type point cloud surfaces, including: Based on the first type of point cloud surface normal that is perpendicular to the other two first type of point cloud surface normals among the three first type of point cloud surface normals, determine the first type of point cloud surface corresponding to the perpendicular first type of point cloud surface normal among the three first type of point cloud surfaces. Based on the relative size between the two included angles, the first type point cloud surface corresponding to the larger included angle and the first type point cloud surface corresponding to the smaller included angle are determined from the other two first type point cloud surfaces. The first type of point cloud surface corresponding to the first calibration surface is determined to be the first type of point cloud surface corresponding to the normal of the first type of point cloud surface perpendicular to it; the first type of point cloud surface corresponding to the second calibration surface is the first type of point cloud surface corresponding to the large included angle; and the first type of point cloud surface corresponding to the third calibration surface is the first type of point cloud surface corresponding to the small included angle.

5. The method according to claim 1, characterized in that, Based on the color features, the point cloud data is segmented and identified to determine the second type of point cloud surface corresponding to each of the three intersecting calibration surfaces, including: The image acquisition device acquires the point cloud data of the calibration phantom, and the corresponding color image and depth image are obtained. Based on the different colors of the three intersecting calibration surfaces, the regions where the three intersecting calibration surfaces are located are determined on the color image. The regions where the three intersecting calibration surfaces are located are mapped to the depth image, and the second type of point cloud surface corresponding to each of the three intersecting calibration surfaces is determined in the point cloud data corresponding to the depth image.

6. The method according to claim 1, characterized in that, The step of obtaining the transformation relationship between the image acquisition device coordinate system and the medical system coordinate system based on the three target normals and the normals of the three intersecting calibration surfaces in the medical system coordinate system includes: Based on the three target normals and the normals of the three intersecting calibration surfaces in the medical system coordinate system, the rotation matrix between the image acquisition device coordinate system and the medical system coordinate system is obtained. Based on the rotation matrix, the transformation relationship between the coordinate system of the image acquisition device and the coordinate system of the medical system is obtained.

7. The method according to claim 6, characterized in that, Based on the rotation matrix, the transformation relationship between the image acquisition device coordinate system and the medical system coordinate system is obtained, including: The point cloud data is rotated using the rotation matrix, and the rotated point cloud data is then fitted. Based on the fitting results, the position of the target corner point in the coordinate system of the image acquisition device is determined; the target corner point is the corner point of the calibration phantom acquired by the image acquisition device. Based on the position of the target corner point in the image acquisition device coordinate system and the medical system coordinate system, the translation matrix between the image acquisition device coordinate system and the medical system coordinate system is obtained; Based on the rotation matrix and the translation matrix, the transformation relationship between the coordinate system of the image acquisition device and the coordinate system of the medical system is obtained.

8. A coordinate system transformation device for a medical system, characterized in that, The device includes: The point cloud acquisition module is used to acquire point cloud data of the calibration phantom at a target location using an image acquisition device; the target location is the position of three intersecting calibration surfaces of the calibration phantom. The segmentation and recognition module is used to segment the point cloud data to obtain three point cloud surfaces, and to determine the point cloud surface corresponding to each of the three intersecting calibration surfaces among the three point cloud surfaces; The normal determination module is used to obtain three target normals based on the point cloud surfaces corresponding to the three intersecting calibration surfaces and the normals of the three point cloud surfaces in the coordinate system of the image acquisition device. The transformation relationship acquisition module is used to obtain the transformation relationship between the image acquisition device coordinate system and the medical system coordinate system based on the three target normals and the normals of the three intersecting calibration surfaces in the medical system coordinate system. Geometric and / or color features of three intersecting calibration surfaces: The geometric features include: the first calibration plane of the three intersecting calibration planes is perpendicular to the second and third calibration planes, and the angle between the second calibration plane and the target coordinate axis is greater than the angle between the third calibration plane and the target coordinate axis; The color feature includes: three intersecting calibration surfaces each having a different color; The point cloud data is segmented to obtain three point cloud surfaces. Among these three point cloud surfaces, the point cloud surfaces corresponding to the three intersecting calibration surfaces are determined. Based on the point cloud surfaces corresponding to the three intersecting calibration surfaces and the normals of the three point cloud surfaces in the image acquisition device coordinate system, three target normals are obtained, including: Based on the geometric features, the point cloud data is segmented and identified to determine the first type of point cloud surface corresponding to each of the three intersecting calibration surfaces. Based on the normals of the first type of point cloud surfaces corresponding to each of the three intersecting calibration surfaces in the coordinate system of the image acquisition device, the target normals of each of the three intersecting calibration surfaces are obtained. Alternatively, based on the color features, the point cloud data is segmented and identified to determine the second type of point cloud surface corresponding to each of the three intersecting calibration surfaces. Based on the normals of the second type of point cloud surfaces corresponding to each of the three intersecting calibration surfaces in the coordinate system of the image acquisition device, the target normals of each of the three intersecting calibration surfaces are obtained. Alternatively, based on the geometric features, the point cloud data is segmented and identified to determine the first type of point cloud surface corresponding to each of the three intersecting calibration surfaces. Based on the normals of the first type of point cloud surfaces corresponding to each of the three intersecting calibration surfaces in the image acquisition device coordinate system, the first type of normals of each of the three intersecting calibration surfaces are obtained. Based on the color features, the point cloud data is segmented and identified to determine the second type of point cloud surface corresponding to each of the three intersecting calibration surfaces. Based on the normals of the second type of point cloud surfaces corresponding to each of the three intersecting calibration surfaces in the image acquisition device coordinate system, the second type of normals of each of the three intersecting calibration surfaces are obtained. The first type of normals and the second type of normals of the same calibration surface are weighted to obtain the target normals of each of the three intersecting calibration surfaces.

9. The apparatus according to claim 8, characterized in that, The segmentation and recognition module is used for: The point cloud data is segmented to obtain three first-class point cloud surfaces; The normals of the three first-type point cloud surfaces in the coordinate system of the image acquisition device are used as the normals of the three first-type point cloud surfaces. Based on the first type point cloud surface normal that is perpendicular to the other two first type point cloud surface normals among the three first type point cloud surface normals, and the relative size between the two included angles, the first type point cloud surface corresponding to each of the three intersecting calibration surfaces is determined among the three first type point cloud surfaces; the two included angles are the angles between the other two first type point cloud surface normals and the target coordinate axis of the image acquisition device coordinate system.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 7.

Images