A calibration method, device, and computer-readable storage medium

Abstract

The present application discloses a calibration method and device, and a computer-readable storage medium, relating to the technical field of data processing, including: providing a first calibration model with calibration points and a surface file of the first calibration model, and obtaining the coordinates of the calibration points in the surface file coordinate system; using the coordinates of the calibration points in the surface file coordinate system and point cloud registration to obtain the coordinates of the calibration points in the depth camera coordinate system; obtaining the coordinates of the calibration points in the optical navigator coordinate system; using the coordinates of the calibration points in the depth camera coordinate system and the coordinates of the calibration points in the optical navigator to obtain a transformation matrix between the depth camera coordinate system and the optical navigator coordinate system; wherein, the first calibration model is a three-dimensional first calibration model with feature information. The present application is used for camera calibration between a depth camera and an optical navigator, making subsequent navigation positioning more accurate and filling the blank of such camera calibration.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and more specifically, to a calibration method and device, and a computer-readable storage medium. Background Art

[0002] A depth camera is a new type of camera that can measure the depth of an object, that is, the relative distance between the object and the origin of the camera. An optical navigator can use near-infrared light (Infrared Radiation, IR) to achieve real-time tracking through additional navigation markers.

[0003] In the current medical field, especially in Transcranial Magnetic Stimulation (TMS) technology, an optical navigator is often used for accurate positioning of the target point. When in use, the patient's head needs to wear a reflective sphere model as the navigation marker of the optical navigator, and it is necessary to ensure that the head and the reflective sphere model remain relatively stationary, which brings discomfort to the patient. To relieve the discomfort of the patient, a depth camera is introduced in the prior art to participate in navigation positioning, and the model on the patient's head is removed.

[0004] Before using an optical navigator and a depth camera, it is necessary to determine the conversion relationship between the coordinate systems of the two, that is, to perform camera calibration. Camera calibration is a commonly used technology in the field of computer vision, and is usually used to determine the conversion relationship between two or more coordinate systems. These coordinate systems may be located inside the camera, that is, the camera internal parameters, which are determined by the internal geometry and optical characteristics of the camera, and mainly include the focal length, the coordinates of the principal point, and the lens distortion parameters, etc.; the camera external parameters refer to the positional relationship between the camera coordinate system and a certain external coordinate system of the camera (such as the world coordinate system), including the rotation matrix and the translation matrix.

[0005] Although there are relatively mature technologies for camera calibration in the above scenarios, the current calibration methods for between an optical navigator and a depth camera are relatively rare, and directly applying the existing calibration methods between an optical navigator and a depth camera will cause a large deviation in the calibration results. Summary of the Invention

[0006] In view of this, this application provides a calibration method and device, and a computer-readable storage medium, which are used to perform camera calibration between a depth camera and an optical navigator, so that subsequent navigation positioning is more accurate, filling the blank of such camera calibration.

[0007] In a first aspect, this application provides a calibration method, which is applied between a depth camera and an optical navigator, and the calibration method includes:

[0008] Provide a first calibration model with calibration points and the surface file of the first calibration model, and obtain the coordinates of the calibration points in the surface file coordinate system;

[0009] Using the coordinates of the calibration points in the surface file coordinate system and point cloud registration, obtain the coordinates of the calibration points in the depth camera coordinate system;

[0010] Obtain the coordinates of the calibration points in the optical navigator coordinate system;

[0011] Using the coordinates of the calibration points in the depth camera coordinate system and the coordinates of the calibration points in the optical navigator, obtain the transformation matrix between the depth camera coordinate system and the optical navigator coordinate system;

[0012] Among them, the first calibration model is a three-dimensional first calibration model with feature information.

[0013] Optionally, among them:

[0014] Providing a first calibration model with calibration points and the surface file of the first calibration model, obtaining the coordinates of the calibration points in the surface file coordinate system includes:

[0015] Provide an initial first calibration model with feature information;

[0016] Mark multiple calibration points on the surface of the initial first calibration model to obtain the first calibration model, and determine the coordinates of each calibration point in the first calibration model coordinate system;

[0017] According to the first calibration model, obtain the surface file of the first calibration model;

[0018] According to the coordinates of the calibration points in the first calibration model coordinate system, obtain the coordinates of the calibration points in the surface file coordinate system;

[0019] Among them, the initial first calibration model is a three-dimensional initial first calibration model.

[0020] Optionally, among them:

[0021] Providing a first calibration model with calibration points and the surface file of the first calibration model, obtaining the coordinates of the calibration points in the surface file coordinate system includes:

[0022] Provide the surface file of the first calibration model with multiple calibration points, and the coordinates of each calibration point in the surface file coordinate system;

[0023] According to the surface file, obtain the first calibration model;

[0024] Among them, the first calibration model is a first calibration model with feature information.

[0025] Optionally, among them:

[0026] Obtaining the coordinates of the calibration points in the depth camera coordinate system by using the coordinates of the calibration points in the surface file coordinate system and point cloud registration includes:

[0027] Obtaining the transformation matrix between the surface file coordinate system and the depth camera coordinate system through point cloud registration;

[0028] Using the coordinates of the calibration points in the surface file coordinate system and the transformation matrix between the surface file coordinate system and the depth camera coordinate system to obtain the coordinates of the calibration points in the depth camera coordinate system.

[0029] Optionally, where:

[0030] Obtaining the transformation matrix between the surface file coordinate system and the depth camera coordinate system through point cloud registration includes:

[0031] When the first calibration model is stationary within the field of view of the depth camera, obtaining the image information of the surface of the first calibration model within the field of view of the depth camera;

[0032] Using the image information to obtain the point cloud information of the surface of the first calibration model within the field of view of the depth camera;

[0033] According to the image information, intercepting the local surface file in the surface file that conforms to the image information;

[0034] Performing registration on the point cloud information and the local surface file to obtain the transformation matrix between the surface file coordinate system and the depth camera coordinate system.

[0035] Optionally, where:

[0036] The optical navigator includes a navigation tool for pointing to the calibration points. Obtaining the coordinates of the calibration points in the optical navigator coordinate system includes:

[0037] Within the field of view of the optical navigator, using the navigation tool to point to the calibration points to obtain the coordinates of the calibration points in the optical navigator coordinate system.

[0038] Optionally, where:

[0039] After obtaining the transformation matrix between the depth camera coordinate system and the optical navigator coordinate system by using the coordinates of the calibration points in the depth camera coordinate system and the coordinates of the calibration points in the optical navigator, the calibration method further includes:

[0040] Providing a second calibration model and fixedly connecting the second calibration model to the depth camera;

[0041] Real-time obtaining the coordinates of the second calibration model in the optical navigator coordinate system;

[0042] Using the coordinates of the second calibration model in the optical navigator coordinate system and the transformation matrix between the depth camera coordinate system and the optical navigator coordinate system, the transformation matrix between the second calibration model coordinate system and the depth camera coordinate system is obtained.

[0043] In a second aspect, the present application further provides a calibration device, including a processor and a memory. The memory is used to store computer program code, and the computer program code includes computer instructions. When the processor executes the computer instructions, the calibration device executes the calibration method described in the first aspect.

[0044] In a third aspect, the present application further provides a computer-readable storage medium, which is characterized by including computer instructions. When the computer instructions run on the calibration device, the calibration device is caused to execute the calibration method described in the first aspect.

[0045] Compared with the prior art, a calibration method, device, and computer-readable storage medium provided by the present application at least achieve the following beneficial effects:

[0046] A calibration method, device, and computer-readable storage medium provided by an embodiment of the present application first obtain the coordinates of the calibration points on the first calibration model in the surface file coordinate system by introducing a first calibration model with a number of calibration points and its surface file. Then, using the coordinates of the calibration points in the surface file coordinate system and by means of a point cloud registration method, the coordinates of the calibration points in the depth camera coordinate system are obtained. Then, the coordinates of the calibration points in the optical navigator coordinate system are obtained. By using the coordinates of the calibration points in the depth camera coordinate system and the coordinates of the calibration points in the optical navigator obtained respectively, the transformation matrix between the depth camera coordinate system and the optical navigator coordinate system is finally obtained, completing the camera calibration between the depth camera and the optical navigator. Based on this, the calibration method, device, and computer-readable storage medium provided by the present application can realize the transformation of points in the world coordinate system between the depth camera coordinate system and the optical navigator coordinate system through the established transformation matrix between the depth camera coordinate system and the optical navigator coordinate system, filling the gap in the current camera calibration method for depth cameras and optical navigators, and avoiding errors caused by inaccurate camera calibration when using the depth camera and the optical navigator in combination for navigation and positioning later, improving the accuracy of subsequent navigation and positioning.

[0047] Among them, a three-dimensional first calibration model with characteristic information is used to replace the commonly used calibration board with a checkerboard in the prior art, and calibration points are set on the first calibration model, providing a calibration object suitable for depth cameras and optical navigators, solving the problem that it is difficult to recognize the calibration board with a checkerboard in depth cameras and optical navigators, making the calibration process more accurate, and thus improving the accuracy and precision of subsequent depth cameras and optical navigators in navigation and positioning.

[0048] In addition, when applied in the TMS field, by adopting the calibration method provided in this application, a depth camera can be introduced to remove the patient's head model, reducing the discomfort of the patient while improving the accuracy of navigation and positioning.

[0049] Of course, when implementing any product of this application, it is not necessarily required to achieve all the above-mentioned technical effects simultaneously.

[0050] Other features and advantages of this application will become clear from the following detailed description of the exemplary embodiments of this application with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The drawings incorporated in and constituting a part of this specification illustrate embodiments of this application and, together with the description, serve to explain the principles of this application.

[0052] Figure 1 The figure shows a flowchart of the calibration method provided by the embodiment of this application;

[0053] Figure 2 The figure shows a schematic diagram of the first calibration model provided by the embodiment of this application;

[0054] Figure 3 The figure shows a schematic diagram of the surface file of the first calibration model provided by the embodiment of this application;

[0055] Figure 4 The figure shows a schematic diagram of the structure of the navigation tool provided by the embodiment of this application;

[0056] Figure 5 The figure shows a schematic diagram of the structure of the second calibration model provided by the embodiment of this application;

[0057] Figure 6 The figure shows a schematic diagram of a structure of the calibration device provided by the embodiment of this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0058] Now, various exemplary embodiments of this application will be described in detail with reference to the accompanying drawings. It should be noted that: unless otherwise specifically stated, the relative arrangements of components and steps, numerical expressions, and numerical values set forth in these embodiments do not limit the scope of this application.

[0059] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way a limitation on this application or its application or use.

[0060] Technologies, methods, and devices known to those of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, such technologies, methods, and devices should be considered as part of the specification.

[0061] In all the examples shown and discussed here, any specific values should be construed as merely exemplary and not as a limitation. Thus, other examples of the exemplary embodiments may have different values.

[0062] It should be noted that like reference numerals and letters refer to like items in the following figures, and thus, once an item is defined in one figure, further discussion thereof is not required in subsequent figures.

[0063] A depth camera is a new type of camera that can measure the depth of an object, i.e., the relative distance of the object from the camera origin. The main principles include binocular vision, structured light, and time-of-flight (TOF) technology, and it can be applied in fields such as 3D modeling, unmanned driving, robot navigation, and motion-sensing games.

[0064] The optical navigator emits near-infrared light (IR) and combines it with a reflective (or luminous) ball made of a specific material to reflect (or directly emit) the infrared light into an infrared receiving sensor, thereby determining the 3D position of the reflective (or luminous) ball. The optical navigator can track the specific position of the reflective ball in real time to achieve precise real-time positioning.

[0065] In current market products, only an optical navigator is used for precise positioning of the target in traditional transcranial magnetic stimulation (TMS) navigation. However, this method requires the patient to wear a reflective ball model on the head, and since the model needs to remain stationary relative to the head, this wearing method usually causes discomfort to the patient. The introduction of a depth camera can eliminate the model worn on the patient's head.

[0066] Camera calibration is a commonly used technique in the field of computer vision, usually used to determine the conversion relationship between two or more coordinate systems. These coordinate systems may be inside the camera, i.e., the camera internal parameters, which are determined by the internal geometry and optical characteristics of the camera and mainly include the focal length, the coordinates of the principal point, and the lens distortion parameters, etc.; the camera external parameters refer to the positional relationship between the camera coordinate system and a certain external coordinate system of the camera (such as the world coordinate system), including the rotation matrix and the translation matrix.

[0067] Camera calibration has a wide range of application scenarios. For example, monocular camera calibration can obtain the internal and external parameters of the camera; binocular camera calibration can determine the conversion relationship between two camera coordinate systems to achieve depth measurement; multi-camera calibration can convert multiple independent camera coordinate systems to the same coordinate system to achieve the unification and expansion of the monitoring range; hand-eye calibration in a robotic arm can unify the camera coordinate system and the robotic arm coordinate system, and so on.

[0068] Although camera calibration has relatively mature technologies in the above scenarios, calibration methods for depth cameras and optical navigators are currently rare. The main difficulties are as follows: Most existing calibration technologies use calibration boards with checkerboards. However, the light sources of depth cameras and optical navigators are both near-infrared light (IR), making the checkerboards blurred in the depth images generated by near-infrared light. Visually, the corner points, which are key features for calibrating ordinary RGB cameras on the checkerboard, cannot be distinguished, resulting in difficulty in distinguishing the checkerboard corner points in depth cameras and optical navigators. Therefore, existing camera calibration technologies are difficult to meet the calibration requirements of depth cameras and optical navigators. In other words, checkerboards are only suitable for calibrating RGB cameras.

[0069] To solve the above technical problems, the embodiments of this application propose a calibration method for camera calibration between depth cameras and optical navigators, making subsequent navigation positioning more accurate and filling the gap in such camera calibration. The following will be described in detail with reference to the accompanying drawings and specific embodiments.

[0070] Figure 1 The following shows a flowchart of a calibration method provided by the embodiments of this application. Figure 2 The following shows a schematic diagram of a first calibration model provided by the embodiments of this application. Figure 3 The following shows a schematic diagram of the surface file of the first calibration model provided by the embodiments of this application.

[0071] Please refer to Figures 1 to 3 , the calibration method provided by the embodiments of this application includes:

[0072] S101, Provide a first calibration model 10 with calibration points 101 and the surface file 11 of the first calibration model 10, and obtain the coordinates of the calibration points 101 in the surface file coordinate system. Among them, the first calibration model 10 is a three-dimensional first calibration model 10 with characteristic information.

[0073] As Figures 1 to 3 shown, to adapt to depth cameras and optical navigators, the embodiments of this application introduce a three-dimensional first calibration model 10 with calibration points 101 and its surface file 11 to replace the traditional two-dimensional calibration board with a checkerboard. Utilizing the characteristic that three-dimensional objects can be clearly presented in the depth images formed by depth cameras and optical navigators, the depth cameras and optical navigators can clearly distinguish the first calibration model 10 and several calibration points 101 thereon, solving the problem that the two-dimensional calibration board with a checkerboard is difficult to recognize in depth cameras and optical navigators, making the calibration process more accurate, and thus improving the accuracy and precision of subsequent depth cameras and optical navigators in navigation positioning.

[0074] In some examples, as Figure 2 and Figure 3As shown, the calibration points 101 of the first calibration model 10 provided in the embodiments of the present application need to be points that can be distinguished in the depth images formed by the depth camera and the optical navigator, that is, the calibration points 101 need to be located on the surface of the first calibration model 10. The existence form of the calibration points 101 can be in the form of depressions or protrusions, etc. Here, only examples are given, and the existence form of the calibration points 101 is not specifically limited.

[0075] In some examples, such as Figures 1 to 3 As shown, since calculations and registrations are required when obtaining the depth camera coordinate system and the optical navigator coordinate system subsequently, the number of calibration points 101 on the first calibration model 10 is at least three, preferably four or more calibration points 101. In order to make the result of camera calibration more accurate, the distribution state of the calibration points 101 on the first calibration model 10 should be as evenly distributed as possible, and try to avoid all the calibration points 101 being coplanar.

[0076] In some examples, such as Figures 1 to 3 As shown, in order to make the registration result obtained by the subsequent point cloud registration method more accurate, the first calibration model 10 provided in the embodiments of the present application is a first calibration model 10 with feature information. And since the point cloud registration method is more sensitive to the feature information of the calibration object, the more obvious the feature information of the first calibration model 10, the more accurate the calibration result. Therefore, the first calibration model 10 is preferably a three-dimensional irregular model with more feature information. If a three-dimensional regular model is selected as the first calibration model 10, it needs to have obvious edge, corner and other feature information. In contrast, a sphere or a disc, etc. are poor marking models because there are no obvious feature points and they cannot be recognized even after rotation.

[0077] In some examples, in order to improve the calibration accuracy, the specific shape of the first calibration model 10 is best combined with the actual usage scenario. For example Figure 2 and Figure 3 As shown, a human head model can be used in the TMS navigation system.

[0078] Exemplarily, such as Figure 2 and Figure 3 As shown, the first calibration model 10 provided in the embodiments of the present application is a three-dimensional human head model, and the calibration points 101 on it exist in the form of depressions and are evenly distributed on the surface of the first calibration model 10.

[0079] It can be understood that the calibration method provided in the embodiments of the present application is not only applicable to the TMS field, but can also be extended to other application scenarios that require precise positioning. The embodiments of the present application only take navigation in the TMS field as an example, and do not specifically limit the application field and scenario.

[0080] Among them, such as Figure 2 andFigure 3 As shown, a first calibration model 10 with calibration points 101 and a surface file 11 of the first calibration model 10 are provided. Obtaining the coordinates of the calibration points 101 in the surface file coordinate system may include:

[0081] Providing an initial first calibration model 10 with feature information;

[0082] Marking a plurality of calibration points 101 on the surface of the initial first calibration model 10 to obtain the first calibration model 10, and determining the coordinates of each calibration point 101 in the first calibration model coordinate system;

[0083] Obtaining the surface file 11 of the first calibration model 10 according to the first calibration model 10;

[0084] Obtaining the coordinates of the calibration points 101 in the surface file coordinate system according to the coordinates of the calibration points 101 in the first calibration model coordinate system;

[0085] Wherein, the initial first calibration model 10 is a three-dimensional initial first calibration model 10.

[0086] Based on this, as Figure 2 and Figure 3 shown, when fabricating the first calibration model 10 with calibration points 101 and its surface file 11, and obtaining the coordinates of the calibration points 101 in the surface file coordinate system, the first calibration model 10 with calibration points 101 can be fabricated first, and then the surface file 11 of the first calibration model 10 can be obtained. Specifically, a plurality of calibration points 101 are marked on the provided three-dimensional initial first calibration model 10 with feature information to obtain the first calibration model 10 that can be clearly recognized by a depth camera and an optical navigator; then, according to the first calibration model 10 and the calibration points 101 thereon, the surface file 11 of the first calibration model 10 and the coordinates of the calibration points 101 in the surface file coordinate system are obtained. Specifically, methods such as scanning can be used to achieve this, such as laser scanning, radiation scanning (such as Computed Tomography, CT), and nuclear magnetic scanning, etc. Here, only examples are given, and the method for obtaining the surface file 11 is not specifically limited.

[0087] It can be understood that, as Figure 2 and Figure 3As shown, in the process of obtaining the coordinates of the calibration point 101 in the surface file coordinate system based on the coordinates of the calibration point 101 in the first calibration model coordinate system, it is not necessary to determine the precise coordinate values of the calibration point 101 in the first model coordinate system. After obtaining the surface file 11 of the first calibration model 10 by means of scanning or other methods, the precise coordinates of the calibration point 101 can be determined in the surface file coordinate system according to the position of the calibration point 101 on the first calibration model 10 and its position presented in the surface file 11. The first calibration model coordinate system and the surface file coordinate system can also be understood as the same coordinate system.

[0088] Exemplarily, as Figure 2 and Figure 3 shown, when the calibration point 101 is a depression, when marking multiple calibration points 101 on the three-dimensional initial first calibration model 10 with feature information, the method of chiseling several calibration points 101 on the initial first calibration model 10 can be adopted. This is only an example here and does not specifically limit the method of making the calibration point 101.

[0089] Optionally, as Figure 2 and Figure 3 shown, providing the first calibration model 10 with the calibration point 101 and the surface file 11 of the first calibration model 10, the process of obtaining the coordinates of the calibration point 101 in the surface file coordinate system may further include:

[0090] Providing the surface file 11 of the first calibration model 10 with multiple calibration points 101, and the coordinates of each calibration point 101 in the surface file coordinate system;

[0091] Obtaining the first calibration model 10 according to the surface file 11;

[0092] wherein, the first calibration model 10 is the first calibration model 10 with feature information.

[0093] Based on this, as Figure 2 and Figure 3 shown, when making the first calibration model 10 with the calibration point 101 and its surface file 11 and obtaining the coordinates of the calibration point 101 in the surface file coordinate system, the surface file 11 of the first calibration model 10 with the calibration point 101 can be made first, and then the first calibration model 10 can be obtained. Specifically, first make the surface file 11 of the first calibration model 10 with the calibration point 101 in the computer, and at the same time obtain the coordinates of each calibration point 101 in the surface file coordinate system. Then, obtain the first calibration model 10 according to the made surface file 11 to prepare for subsequent registration and coordinate transformation.

[0094] In some examples, when making the surface file of the first calibration model with calibration points, drawing software such as CAD can be used for production. When obtaining the first calibration model according to the made surface file, technologies such as 3D printing can be used for production. Here is only an example and is not specifically limited.

[0095] Exemplarily, as Figure 2 and Figure 3 shown, in the first calibration model 10 and its surface file 11 provided in the present application, Figure 2 the calibration points 101 with numbers on the first calibration model 10 shown in Figure 3 and the calibration points 101 represented by depressions on the surface file 11 shown in

[0096] S102. Using the coordinates of the calibration points in the surface file coordinate system and point cloud registration, obtain the coordinates of the calibration points in the depth camera coordinate system.

[0097] Specifically, as Figures 1 to 3 shown, using the coordinates of the calibration points 101 in the surface file coordinate system and point cloud registration, obtaining the coordinates of the calibration points 101 in the depth camera coordinate system may include:

[0098] Obtain the transformation matrix between the surface file coordinate system and the depth camera coordinate system through point cloud registration; specifically, it may include:

[0099] When the first calibration model 10 is stationary within the field of view of the depth camera, obtain the image information of the surface of the first calibration model 10 within the field of view of the depth camera;

[0100] Using the image information, obtain the point cloud information of the surface of the first calibration model 10 within the field of view of the depth camera;

[0101] According to the image information, intercept the local surface file in the surface file 11 that conforms to the image information;

[0102] Perform registration on the point cloud information and the local surface file to obtain the transformation matrix between the surface file coordinate system and the depth camera coordinate system.

[0103] Based on this, as Figures 1 to 3As shown, when obtaining the coordinates of the calibration point 101 in the depth camera coordinate system, first, point cloud registration can be used to obtain the transformation matrix between the surface file coordinate system and the depth camera coordinate system. Specifically, first, place the first calibration model 10 with the calibration point 101 in the field of view of the depth camera for shooting. When the first calibration model 10 is stationary within the field of view of the depth camera, take a shot to obtain the image information of the surface of the first calibration model 10 within the depth camera's field of view. That is, when taking the image of the first calibration model 10, only the surface image of the part of the first calibration model 10 facing the depth camera within the depth camera's field of view needs to be taken, which is the obtained image information. Then, using the obtained image information, the point cloud information of the surface of the first calibration model 10 within the depth camera's field of view can be obtained through the depth camera. That is, the range of the obtained point cloud information is also the same as the image information, which is the surface point cloud of the part of the first calibration model 10 facing the depth camera within the depth camera's field of view, which is the obtained point cloud information. Then, according to the obtained image information, the part of the surface file 11 with the same range as the image information and the point cloud information is intercepted to obtain a local surface file. Then, the point cloud information and the local surface file are subjected to point cloud registration to obtain the transformation matrix between the surface file coordinate system and the depth camera coordinate system. It can be expressed by the formula as follows:

[0104] T_marker_depth = ICP(Cloud_marker,MARKER_surface)

[0105] Among them, T_marker_depth is the transformation matrix between the surface file coordinate system and the depth camera coordinate system, that is, the result of point cloud registration. Cloud_marker is the point cloud information, MARKER_surface is the surface file, and the Iterative Closest Point (ICP) is the specific method of point cloud registration selected in this embodiment of the present application. Here, only an example is given, and it is not limited to ICP only. It can also be methods such as Fast Point Feature Histogram (FPFH), Super 4 -Points Congruent Sets (Super4PCS), Robust Point Matching (RPM), Kernel Correlation (KC), Coherent Point Drift (CPD), Normal Distribution Transform (NDT), etc. Registration methods are not listed one by one here.

[0106] Exemplarily, the point cloud registration can also use the FRFH+ICP registration method. First, perform rough registration of the point cloud, and then perform fine registration of the point cloud to obtain a more accurate point cloud registration result.

[0107] After that, using the coordinates of the calibration points in the surface file coordinate system and point cloud registration to obtain the coordinates of the calibration points in the depth camera coordinate system further includes: using the coordinates of the calibration points in the surface file coordinate system and the transformation matrix between the surface file coordinate system and the depth camera coordinate system to obtain the coordinates of the calibration points in the depth camera coordinate system. Based on this, through the transformation matrix between the surface file coordinate system and the depth camera coordinate system obtained previously, and the coordinates of the calibration points in the surface file coordinate system, the coordinates of the calibration points can be transferred from the surface file coordinate system to the depth camera coordinate system to obtain the coordinates of the calibration points in the depth camera coordinate system. It can be expressed by the formula as:

[0108] Depth_points = MARKER_points * T_marker_depth

[0109] Among them, MARKER_points are the coordinates of the calibration points in the surface file coordinate system, T_marker_depth is the transformation matrix between the surface file coordinate system and the depth camera coordinate system, and Depth_points are the coordinates of the calibration points in the depth camera coordinate system. By multiplying the coordinates MARKER_points of each calibration point with T_marker_depth through the multiplication of a vector and a matrix, the coordinates Depth_points of each calibration point in the depth camera coordinate system can be obtained.

[0110] Figure 4 The following shows the structural schematic diagram of the navigation tool provided by the embodiment of the present application.

[0111] S103, obtain the coordinates of the calibration points in the optical navigator coordinate system.

[0112] Specifically, as Figures 2 to 4 shown, the optical navigator includes a navigation tool 12 for pointing to the calibration points. Obtaining the coordinates of the calibration points 101 in the optical navigator coordinate system includes: within the field of view of the optical navigator, using the navigation tool 12 to point to the calibration points 101 to obtain the coordinates of the calibration points 101 in the optical navigator coordinate system. Based on this, using the characteristic that the optical navigator can obtain the coordinates of the points pointed to by the navigation tool 12, the coordinates of the calibration points 101 in the optical navigator coordinate system, denoted as Navigation_points, can be obtained by pointing the navigation tool 12 to each calibration point 101 on the first calibration model 10.

[0113] Among them, as Figure 4As shown, the navigation tool 12 has a plurality of calibration balls 121 for reflecting IR light, and the plurality of calibration balls together constitute a navigation model. The number of calibration balls 121 varies according to the selected optical navigator and will not be listed one by one here. Commonly used navigation tools 12 have 3 to 5 calibration balls 121.

[0114] Exemplarily, as Figure 4 shown, the navigation tool 12 selected in the embodiment of the present application has 4 calibration balls 121.

[0115] In some examples, the positional relationship among the depth camera, the first calibration model, and the optical navigator only needs to satisfy that the first calibration model faces the depth camera within a certain distance range, so that the depth camera can obtain a clear point cloud image of the first calibration model, and the optical navigator is placed at a position where the first calibration model can be recognized.

[0116] S104. Using the coordinates of the calibration points in the depth camera coordinate system and the coordinates of the calibration points in the optical navigator, obtain the transformation matrix between the depth camera coordinate system and the optical navigator coordinate system. It can be expressed by the formula:

[0117] T_depth_navigation = SVD(Depth_points, Navigation_points)

[0118] Among them, Depth_points are the coordinates of the calibration points in the depth camera coordinate system, Navigation_points are the coordinates of the calibration points in the optical navigator coordinate system, T_depth_navigation is the transformation matrix between the depth camera coordinate system and the optical navigator coordinate system, that is, the result of camera calibration in the embodiment of the present application. Singular Value Decomposition (SVD) is an algorithm selected in the embodiment of the present application to calculate the transformation matrix between the depth camera coordinate system and the optical navigator coordinate system. Here, only the commonly used SVD algorithm is used as an example, and the calculation method of the transformation matrix is not specifically limited.

[0119] The current camera calibration requires that the calibrated coordinate systems do not undergo relative displacement after calibration. If displacement occurs, recalibration is required; if the depth camera and the optical navigator are far apart, slight relative displacement will inevitably occur between the two during use, which will ultimately affect the positioning accuracy of the entire system.

[0120] Figure 5 Shown is the structural schematic diagram of the second calibration model provided by the embodiment of the present application.

[0121] To avoid displacement of the depth camera and the optical navigator after camera calibration, which reduces the accuracy of the entire system, and in combination with the characteristics of the optical navigator, in some examples, such as Figure 5 shown, after obtaining the transformation matrix between the depth camera coordinate system and the optical navigator coordinate system by using the coordinates of the calibration points in the depth camera coordinate system and the coordinates of the calibration points in the optical navigator, the calibration method further includes:

[0122] Providing a second calibration model 14 and fixedly connecting the second calibration model 14 to the depth camera;

[0123] Obtaining the coordinates of the second calibration model 14 in the optical navigator coordinate system in real time;

[0124] Using the coordinates of the second calibration model 14 in the optical navigator coordinate system and the transformation matrix between the depth camera coordinate system and the optical navigator coordinate system to obtain the transformation matrix between the second calibration model coordinate system and the depth camera coordinate system.

[0125] Specifically, as Figure 4 and Figure 5 shown, after obtaining the transformation matrix between the depth camera coordinate system and the optical navigator coordinate system, a second calibration model 14 can be fixed on the depth camera. Then, by virtue of the optical navigator's characteristic of being able to obtain the coordinates of the point pointed to by the navigation tool 12 with the aid of the navigation tool 12, the coordinates of the second calibration model 14 in the optical navigator coordinate system can be obtained in real time. At the same time, the transformation matrix between the second calibration model coordinate system and the optical navigator coordinate system can also be obtained. Therefore, relative displacement between the optical navigator and the 3D model is allowed, enhancing the flexibility of the entire system; finally, using the coordinates of the second calibration model 14 in the optical navigator coordinate system and the transformation matrix between the depth camera coordinate system and the optical navigator coordinate system, the transformation matrix between the second calibration model coordinate system and the depth camera coordinate system can be obtained. It can be expressed by the formula as:

[0126] T_depth_model = T_depth_navigation * T_navigation_model

[0127] where T_depth_navigation is the transformation matrix between the depth camera coordinate system and the optical navigator coordinate system, T_navigation_model is the transformation relationship between the second calibration model coordinate system and the optical navigator coordinate system, and T_depth_model is the transformation matrix between the second calibration model coordinate system and the depth camera coordinate system, which is the final calibration result.

[0128] Based on this, as Figure 5As shown in the figure, introducing the second calibration model 14 during the calibration process can convert the calibration between the depth camera and the optical navigator into the calibration between the depth camera and the second calibration model 14. Since the depth camera and the second calibration model 14 are fixedly connected, the relative displacement is relatively stable, improving the robustness of the system accuracy. At the same time, after the calibration is completed, the optical navigator can be flexibly placed relative to the depth camera, as long as the second calibration model 14 is within the field of view of the optical navigator, improving the flexibility of the system. In addition, due to the functions of the optical navigator itself, the conversion relationship between the second calibration model coordinate system and the optical navigator coordinate system can be obtained through the optical navigator. Therefore, the final camera calibration result can be arbitrarily converted between the depth camera and the optical navigator or the second calibration model 14.

[0129] Among them, as Figure 5 shown, the second calibration model 14 can be a three-dimensional calibration model provided with a 3D calibration sphere 141 that can reflect IR light.

[0130] Figure 6 As shown in the figure is a schematic structural diagram of a calibration device provided by an embodiment of the present application.

[0131] Based on the same inventive concept, as Figure 6 shown, the present application further provides a calibration device 200, including a processor 201 and a memory 202. The memory 202 is used to store computer program code 203. The computer program code 203 includes computer instructions. When the processor 201 executes the computer instructions, the calibration device 200 executes each step performed by the calibration device in the method flow shown in the above method embodiment.

[0132] Based on the same inventive concept, the present application further provides a computer-readable storage medium, which is characterized by including computer instructions. When the computer instructions run on the calibration device, the calibration device is enabled to execute each step performed by the calibration device in the method flow shown in the above method embodiment.

[0133] Based on the same inventive concept, the embodiment of the present invention further provides a computer program product. The computer program product includes computer instructions. When the computer instructions run on the calibration device, the calibration device is enabled to execute each step performed by the calibration device in the method flow shown in the above method embodiment.

[0134] In summary, a calibration method, device, and computer-readable storage medium provided by the present application at least achieve the following beneficial effects:

[0135] A calibration method, device, and computer-readable storage medium provided by an embodiment of the present application first obtain the coordinates of calibration points on a first calibration model in the surface file coordinate system by introducing a first calibration model with a number of calibration points and its surface file. Then, the coordinates of the calibration points in the depth camera coordinate system are obtained by using the coordinates of the calibration points in the surface file coordinate system and with the help of a point cloud registration method. Next, the coordinates of the calibration points in the optical navigator coordinate system are obtained. By using the coordinates of the calibration points in the depth camera coordinate system and the coordinates of the calibration points in the optical navigator respectively obtained, the transformation matrix between the depth camera coordinate system and the optical navigator coordinate system is finally obtained, completing the camera calibration between the depth camera and the optical navigator. Based on this, the calibration method, device, and computer-readable storage medium provided by the present application can realize the transformation of points in the world coordinate system between the depth camera coordinate system and the optical navigator coordinate system through the established transformation matrix between the depth camera coordinate system and the optical navigator coordinate system, filling the blank of the current camera calibration method for depth cameras and optical navigators, so that errors caused by inaccurate camera calibration can be avoided when the depth camera and the optical navigator are used in combination for navigation and positioning later, and the accuracy of subsequent navigation and positioning is improved.

[0136] Among them, a three-dimensional first calibration model with feature information is used to replace the calibration board with a checkerboard commonly used in the prior art, and calibration points are set on the first calibration model, providing a calibration object suitable for depth cameras and optical navigators, solving the problem that the calibration board with a checkerboard is difficult to recognize in depth cameras and optical navigators, making the calibration process more accurate, and thus improving the accuracy and precision of subsequent depth cameras and optical navigators in navigation and positioning.

[0137] In addition, when applied in the TMS field, by using the calibration method provided by the present application, a depth camera can be introduced to remove the patient's head model, reducing the discomfort of the patient while improving the accuracy of navigation and positioning.

[0138] Although some specific embodiments of the present application have been described in detail by way of examples, those skilled in the art should understand that the above examples are only for illustration and not for limiting the scope of the present application. Those skilled in the art should understand that the above embodiments can be modified without departing from the scope and spirit of the present application. The scope of the present application is defined by the appended claims.

Claims

1. A calibration method, characterized in that, Applied between a depth camera and an optical navigator, the calibration method includes: Providing a first calibration model with calibration points and a surface file of the first calibration model, and obtaining the coordinates of the calibration points in the surface file coordinate system; the first calibration model is a three-dimensional human head model, and the calibration points on the three-dimensional human head model exist in the form of depressions and are evenly distributed on the surface of the first calibration model; Using the coordinates of the calibration points in the surface file coordinate system and point cloud registration to obtain the coordinates of the calibration points in the depth camera coordinate system; Obtaining the coordinates of the calibration points in the optical navigator coordinate system; Using the coordinates of the calibration points in the depth camera coordinate system and the coordinates of the calibration points in the optical navigator to obtain the transformation matrix between the depth camera coordinate system and the optical navigator coordinate system; Wherein, the first calibration model is a three-dimensional first calibration model with feature information.

2. The calibration method according to claim 1, wherein The step of providing a first calibration model with calibration points and a surface file of the first calibration model, and obtaining the coordinates of the calibration points in the surface file coordinate system includes: Providing an initial first calibration model with feature information; Marking a plurality of calibration points on the surface of the initial first calibration model to obtain a first calibration model, and determining the coordinates of each calibration point in the first calibration model coordinate system; Obtaining the surface file of the first calibration model according to the first calibration model; Obtaining the coordinates of the calibration points in the surface file coordinate system according to the coordinates of the calibration points in the first calibration model coordinate system; Wherein, the initial first calibration model is a three-dimensional initial first calibration model.

3. The calibration method according to claim 1, wherein The step of providing a first calibration model with calibration points and a surface file of the first calibration model, and obtaining the coordinates of the calibration points in the surface file coordinate system includes: Providing a surface file of a first calibration model with a plurality of calibration points, and the coordinates of each calibration point in the surface file coordinate system; Obtaining the first calibration model according to the surface file; Wherein, the first calibration model is a first calibration model with feature information.

4. The calibration method according to claim 1, wherein The step of using the coordinates of the calibration points in the surface file coordinate system and point cloud registration to obtain the coordinates of the calibration points in the depth camera coordinate system includes: Obtaining the transformation matrix between the surface file coordinate system and the depth camera coordinate system through point cloud registration; Using the coordinates of the calibration points in the surface file coordinate system and the transformation matrix between the surface file coordinate system and the depth camera coordinate system to obtain the coordinates of the calibration points in the depth camera coordinate system.

5. The calibration method according to claim 4, wherein The step of obtaining the transformation matrix between the surface file coordinate system and the depth camera coordinate system through point cloud registration includes: When the first calibration model is stationary within the field of view of the depth camera, obtaining the image information of the surface of the first calibration model within the field of view of the depth camera; Using the image information to obtain the point cloud information of the surface of the first calibration model within the field of view of the depth camera; According to the image information, intercepting the local surface file in the surface file that conforms to the image information; Register the point cloud information and the local surface file to obtain the transformation matrix between the surface file coordinate system and the depth camera coordinate system.

6. The calibration method according to claim 1, wherein The optical navigator includes a navigation tool for pointing to the calibration point, and the obtaining of the coordinates of the calibration point in the optical navigator coordinate system includes: Within the field of view of the optical navigator, use the navigation tool to point to the calibration point and obtain the coordinates of the calibration point in the optical navigator coordinate system.

7. The calibration method according to any one of claims 1 to 6, characterized in that, After obtaining the transformation matrix between the depth camera coordinate system and the optical navigator coordinate system by using the coordinates of the calibration point in the depth camera coordinate system and the coordinates of the calibration point in the optical navigator, the calibration method further includes: Provide a second calibration model and fixedly connect the second calibration model to the depth camera; Obtain the coordinates of the second calibration model in the optical navigator coordinate system in real time; Use the coordinates of the second calibration model in the optical navigator coordinate system and the transformation matrix between the depth camera coordinate system and the optical navigator coordinate system to obtain the transformation matrix between the second calibration model coordinate system and the depth camera coordinate system.

8. A calibration device, characterized in that, It includes a processor and a memory. The memory is used to store computer program code. The computer program code includes computer instructions. When the processor executes the computer instructions, the calibration device executes the calibration method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, It includes computer instructions. When the computer instructions run on the calibration device, the calibration device is caused to execute the calibration method according to any one of claims 1 to 7.

Images