Calibration method, tracking scanning system and storage medium

Abstract

The application provides a calibration method, a tracking scanning system and a storage medium. The tracking scanning system comprises a tracker and a scanner. The method comprises: tracking a first marker and a second marker on the scanner by using the tracker to obtain a first marker point set of the first marker in a tracking coordinate system of the tracker and a second marker point set of the second marker in the tracking coordinate system; wherein the relative positions between the scanner and the first marker and the second marker are fixed; and based on the first marker point set and the second marker point set, a target conversion relationship between a local coordinate system corresponding to the first marker and a scanning coordinate system of the scanner is calibrated. The above method can reduce the calibration cost and improve the calibration efficiency.

Description

Technical Field

[0001] This application relates to the field of 3D scanning, and more particularly to a calibration method, a tracking scanning system, and a storage medium. Background Technology

[0002] Currently, tracking scanning systems typically consist of a laser tracker (or optical tracker) and a handheld scanner. During calibration, hand-eye calibration is generally achieved by the tracker measuring marker points on the ball cage of the scanner and marker points on the hand-eye calibrator. However, this type of calibration is cumbersome and inefficient. Summary of the Invention

[0003] This application discloses a calibration method, a tracking scanning system, and a storage medium, which solves the technical problem of low calibration efficiency in existing calibration methods.

[0004] This application provides a calibration method for a tracking scanning system, the tracking scanning system including a tracker and a scanner. The method includes: using the tracker to track a first marker and a second marker on the scanner, obtaining a first set of marker points for the first marker in the tracking coordinate system of the tracker, and a second set of marker points for the second marker in the tracking coordinate system; wherein the relative positions of the scanner with respect to the first marker and the second marker are fixed; and based on the first set of marker points and the second set of marker points, calibrating the target transformation relationship between the local coordinate system corresponding to the first marker and the scanning coordinate system of the scanner.

[0005] In some embodiments of this application, the method further includes: obtaining a first reference coordinate set of the first marker in the local coordinate system; obtaining a second reference coordinate set of the second marker in the scan coordinate system; and calibrating the target transformation relationship based on the first marker set, the second marker set, the first reference coordinate set, and the second reference coordinate set.

[0006] In some embodiments of this application, the step of calibrating the target transformation relationship based on the first set of marker points, the second set of marker points, the first set of reference coordinates, and the second set of reference coordinates includes: determining a first transformation relationship based on the first set of marker points and the first set of reference coordinates, wherein the first transformation relationship is the relative positional relationship between the local coordinate system and the tracking coordinate system; determining a second transformation relationship based on the second set of marker points and the second set of reference coordinates, wherein the second transformation relationship is the relative positional relationship between the scanning coordinate system and the tracking coordinate system; and determining the target transformation relationship based on the first transformation relationship and the second transformation relationship.

[0007] In some embodiments of this application, the first set of marker points includes a first subset of multiple frames, and the second set of marker points includes a second subset of multiple frames. The method further includes: generating a first transformation relationship corresponding to multiple frames based on the first subset of multiple frames and the first reference coordinate set; generating a second transformation relationship corresponding to multiple frames based on the second subset of multiple frames and the second reference coordinate set; and calculating the optimal solution for the first transformation relationship and the second transformation relationship corresponding to multiple frames using a hand-eye calibration algorithm to determine the target transformation relationship.

[0008] In some embodiments of this application, the first set of marker points includes a first subset of multiple frames, and the second set of marker points includes a second subset of multiple frames. The method further includes: determining a candidate relationship for the i-th frame based on the first subset of the i-th frame, the second subset of the i-th frame, the first reference coordinate set, and the second reference coordinate set; i is a positive integer; determining a third transformation relationship for the i-th frame based on the first subset of the i-th frame and the second subset of the i-th frame; the third transformation relationship for the i-th frame is the relative positional relationship between the first marker and the second marker in the i-th frame; calculating a first difference between the third transformation relationship for the i-th frame and the candidate relationship for the i-th frame; and fusing multiple candidate relationships corresponding to the first difference that is less than or equal to a preset difference to complete the calibration of the target transformation relationship.

[0009] In some embodiments of this application, the step of calibrating the target transformation relationship between the local coordinate system corresponding to the first marker and the scanning coordinate system of the scanner based on the first marker set and the second marker set includes: determining a fourth transformation relationship based on the first marker set and the second marker set, wherein the fourth transformation relationship is the relative positional relationship between the first marker and the second marker determined based on the first marker set and the second marker set; evaluating the calibration accuracy of the target transformation relationship using the fourth transformation relationship; if the target transformation relationship meets the preset calibration accuracy, then the calibration of the target transformation relationship is completed; if the target transformation relationship does not meet the preset calibration accuracy, then the target transformation relationship is recalibrated.

[0010] In some embodiments of this application, the method further includes: calculating a second difference between the fourth transformation relationship and the target transformation relationship; if the second difference is less than or equal to a preset threshold, determining that the target transformation relationship meets a preset calibration accuracy, and completing the calibration of the target transformation relationship; if the second difference is greater than the preset threshold, determining that the target transformation relationship does not meet the preset calibration accuracy, and then recalibrating the target transformation relationship.

[0011] In some embodiments of this application, the method further includes: acquiring data points in the first set of marker points and data points in the second set of marker points; if the number of data points in the first set of marker points does not meet a first preset number, and / or the number of data points in the second set of marker points does not meet a second preset number, then re-tracking the scanner using the tracker; if the number of data points in the first set of marker points meets the first preset number, and the number of data points in the second set of marker points meets the second preset number, then performing the step of calibrating the target transformation relationship between the local coordinate system corresponding to the first marker and the scanning coordinate system of the scanner based on the first set of marker points and the second set of marker points.

[0012] In some embodiments of this application, after calibrating the target transformation relationship between the local coordinate system corresponding to the first marker and the scanning coordinate system of the scanner, the method further includes: scanning a specified object using the scanner to obtain point cloud data of the surface of the specified object; tracking the first marker using the tracker to obtain the pose of the first marker; and transforming the point cloud data into the tracking coordinate system based on the pose of the first marker and the target transformation relationship.

[0013] This application also provides a tracking scanning system, the tracking scanner system comprising: a scanner for moving a first marker and a second marker during calibration, wherein the relative positions of the scanner with respect to the first marker and the second marker are fixed respectively; a tracker for tracking the first marker and the second marker, thereby measuring a first set of marker points of the first marker and a second set of marker points of the second marker; and an electronic device for calibrating the target transformation relationship between the local coordinate system corresponding to the first marker and the scanning coordinate system of the scanner based on the first set of marker points and the second set of marker points.

[0014] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the calibration method described above.

[0015] In the calibration method provided in this application, a first marker and a second marker are provided on the scanner. The relative positions of the scanner with the first marker and the second marker are fixed, respectively. This allows calibration to be completed without adding additional calibration devices (such as hand-eye calibrators), reducing calibration costs. Specifically, a tracker tracks the first and second markers on the scanner to obtain a first set of marker points for the first marker in the tracking coordinate system and a second set of marker points for the second marker in the tracking coordinate system, providing data for subsequent calibration. Therefore, based on the first and second marker point sets, the target transformation relationship between the local coordinate system corresponding to the first marker and the scanner's scanning coordinate system can be calibrated, simultaneously ensuring both calibration accuracy and efficiency. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the structure of the trace scanning system provided in the embodiments of this application.

[0017] Figure 2 This is a schematic diagram showing the positional relationship between the scanner, the first marker, and the second marker provided in an embodiment of this application.

[0018] Figure 3 This is a flowchart of the calibration method provided in the embodiments of this application.

[0019] Figure 4 This is a schematic diagram of the tracker and scanner provided in the embodiments of this application.

[0020] Figure 5 This is a flowchart of a calibration method provided in another embodiment of this application.

[0021] Figure 6 This is a flowchart of a calibration method provided in another embodiment of this application.

[0022] Figure 7 This is a flowchart of the calibration accuracy detection of the target transformation relationship provided in the embodiments of this application.

[0023] Figure 8 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0024] For ease of understanding, some concepts related to the embodiments of this application are illustrated and explained by way of example for reference.

[0025] It should be noted that in this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and drawings of this application are used to distinguish similar objects, not to describe a specific order or sequence.

[0026] In the current field of 3D scanning, common tracking scanning systems typically consist of two main parts: a laser tracker (or optical tracker) that serves as a global coordinate reference, and a handheld scanner used to actually acquire 3D data of the object's surface. The core objective of the entire system calibration is to accurately determine the transformation relationship between the scanner's coordinate system and the tracker's coordinate system (i.e., the "hand-eye relationship").

[0027] Existing calibration procedures rely heavily on physical aids. Specifically, a dedicated hand-eye calibration device is required during calibration. This device is equipped with markers (such as a reflective target sphere) with known precise geometric relationships. During calibration, the laser tracker needs to simultaneously observe and measure two types of points: one type is the markers within the tracking target (commonly a "ball cage" structure) fixed to the handheld scanner; the other type is the markers distributed on the hand-eye calibration device. By repeatedly changing the relative pose of the scanner and the calibration device and simultaneously recording multiple sets of corresponding 3D coordinates of the markers, the required coordinate transformation matrix can finally be calculated.

[0028] This traditional method requires operators to repeatedly and precisely position the calibrator and scanner to ensure that the marker points on both are simultaneously and stably locked by the tracker. The entire process involves many steps, is time-consuming, and requires a certain level of operator experience. Therefore, the overall efficiency of this calibration method is not high, which to some extent affects the rapid deployment and ease of use of the scanning system.

[0029] Therefore, in order to better understand the calibration method, tracking scanning system and storage medium provided in the embodiments of this application, the structure of the tracking scanning system provided in this application will be described first below.

[0030] Figure 1 This is a schematic diagram of the tracer scanning system provided in an embodiment of this application. Figure 1 As shown, the tracking scanning system includes an electronic device 10, a tracker 20, and a scanner 30.

[0031] Electronic device 10 is communicatively connected to tracker 20 and scanner 30, respectively. The communication connection methods can include wired and wireless communication. Wired communication connections can include one or more of the following: Universal Serial Bus (USB), Controller Area Network (CAN), etc. Wireless communication connections can include one or more of the following: Wireless Fidelity (Wi-Fi), Bluetooth (BT), mobile communication networks, Frequency Modulation (FM), Near Field Communication (NFC), Infrared (IR), etc.

[0032] Electronic device 10 may include devices with communication functions such as laptops, tablets, programmable logic controllers (PLCs), and human-machine interfaces (HMIs) with touch input capabilities, or devices simulated by virtual machines or simulators. Electronic device 10 is used to receive data sent by tracker 20 and scanner 30 respectively, and is also used for calibration between different coordinate systems.

[0033] Tracker 20 can be an optical tracker or a laser tracker.

[0034] The scanner 30 may include, but is not limited to, oral scanning devices, facial scanning devices, CT (Computed Tomography) scanning devices or CBCT (Cone Beam Computer Tomography) scanning devices, professional scanners, industrial scanners, etc. Oral scanning devices include intraoral scanners and extraoral scanners. The scanner 30 may be a handheld scanning device or a fixed scanning device. The scanner 30 can measure a specified object to achieve three-dimensional reconstruction. For example, the specified object may be teeth, a face, a human body, industrial products, industrial equipment, cultural relics, works of art, prostheses, medical instruments, buildings, and other human bodies, objects, or scenes. This application does not impose specific limitations in this regard.

[0035] The scanner 30 may include a first marker 310 and a second marker 320.

[0036] The first marker 310 can be a rigid frame composed of multiple marker disks, with multiple marker points set on each disk. The scanner 30 and the first marker 310 can be an integral structure or two detachable independent devices. The relative position between the scanner 30 and the first marker 310 is fixed.

[0037] The second marker 320 can be an LED light, target point, or marker point set on the scanner 30. The second marker 320 can be directly attached to the scanner 30 body, or it can extend from the body via a crossbar or other support to effectively capture the target point on the second marker 320. The relative position between the second marker 320 and the scanner 30 is fixed.

[0038] The tracker 20 can identify the marker points of the first marker 310 and the marker points corresponding to the second marker 320 through a binocular camera system to achieve tracking of the scanner 30.

[0039] To better understand the positional relationship between the scanner 30 and the first marker 310 and the second marker 320, the following will be combined with... Figure 2 Describe it.

[0040] The first marker 310 can be a rigid frame composed of multiple marker disks, with multiple marker points set on each disk. The rigid frame can be as follows: Figure 2 The ball cage shown can have markings on the disc as follows: Figure 2 The disc on the ball cage shown can be multiple points on the disc (e.g., ...). Figure 2 As shown in M1). Figure 2 The first marker 310 shown can be kept at a fixed distance from the scanner 30 body by means of a crossbar or other support. This application does not limit the number of marker disks of the first marker 310 or the number of markers on each marker disk.

[0041] The second marker 320 can be composed of multiple target points (such as...) Figure 2 A rigid array consisting of M2 (as shown) Figure 2 The shape and number of target points shown are for illustrative purposes only and can be set according to actual conditions during application; this application does not impose any restrictions on them. The second marker 320 can be directly pasted onto the body of the scanner 30. To ensure the accuracy of subsequent calibration, the number of target points can be set to no less than 4. Experiments have shown that a number of target points greater than or equal to 8 is optimal.

[0042] In the subsequent calibration process, the relative positions of the scanner 30 with the first marker 310 and the second marker 320 are fixed.

[0043] The above illustration Figure 1 This is merely an example of a tracking scanning system and does not constitute a limitation on tracking scanning systems. It may include more or fewer components than illustrated, or combine certain components, or use different components, such as... Figure 1 The electronic device 10 shown may also include input / output devices, network access devices, etc.

[0044] Figure 3 This is a flowchart of the calibration method provided in the embodiments of this application, which is applied in, for example... Figure 1 The flowchart illustrates a tracking scanning system. Depending on the specific requirements, the order of the steps can be changed, and some steps can be omitted.

[0045] Step S301: Use a tracker to track the first marker and the second marker on the scanner to obtain the first marker point set of the first marker in the tracking coordinate system of the tracker, and the second marker point set of the second marker in the tracking coordinate system.

[0046] In some embodiments of this application, the first marker may be a disk of multiple marker points on the scanner, and the second marker may be a target point, an LED light, etc. Before calibration, the system installation environment can be checked and ensured to be stable to avoid environmental vibration and strong light interference. Subsequently, the tracker, scanner, and electronic equipment are started sequentially.

[0047] During the calibration process, combined with Figure 4 As shown, within the tracking range of the tracker 20, the orientation and position of the scanner 30 can be randomly changed, and the tracker 30 measures the first marker 310 and the second marker 320 on the scanner 30 in real time.

[0048] The electronic device can be equipped with dedicated system control and data processing software, and connects to the tracker and scanner via independent communication interfaces to establish real-time communication. The electronic device receives from the tracker a first set of marker points corresponding to a first marker and a second set of marker points corresponding to a second marker on the scanner. Both the first and second marker point sets are located on the tracking coordinate system corresponding to the tracker. Additionally, the electronic device receives scan data of a specified object from the scanner. For example, the specified object can be teeth, a face, a body, industrial products, industrial equipment, cultural relics, works of art, prostheses, medical instruments, buildings, or other human bodies, objects, or scenes. The scan data of the specified object includes 3D point cloud data or surface data.

[0049] In some embodiments of this application, since the scanner needs to rely on a robotic arm or be held by the user to complete the movement, the tracker may be obstructed during the tracking process of the scanner. Therefore, the electronic device can perform quality inspection on the first set of marker points and the second set of marker points after obtaining them.

[0050] In one example, assume that the first preset number of marker points corresponding to the first marker is set to 10, and the first marker point set is the result of a single frame measurement. Calculate the number of data points in the first marker point set. If the number of data points in the first marker point set is 6, it means that the number of data points in the first marker point set is less than the first preset number. If the number of data points in the first marker point set does not meet the first preset number, the tracker can be used to re-track the scanner.

[0051] In another example, assuming the first set of marker points is the result of multi-frame measurements, the first set of marker points may include multiple first subsets, each representing the result of a single-frame measurement of the first marker. If the first set of marker points is the result of multi-frame measurements, the first preset number may be a baseline number set for each first subset. For example, the first preset number of marker points corresponding to the first marker may be set to 10, representing the minimum number of marker points expected to be measured in a single frame under ideal conditions. The number of data points in each first subset is calculated. If the number of data points in any first subset is 6, it indicates that the number of data points in that first subset is less than the first preset number. This indicates that the number of data points in that first subset does not meet the first preset number, and that first subset can be set as invalid data. Multiple first subsets that meet the first preset number are acquired. If the number of multiple first subsets that meet the first preset number does not meet the target number, the tracker is reused to reacquire the first set of marker points, i.e., multiple first subsets are reacquired.

[0052] Specifically, since both the first and second markers are mounted on the scanner, and the second marker can be a ball-cage structure, it may be obstructed by the robotic arm and / or the user's hand and / or the first marker. Therefore, to improve the accuracy of subsequent calibration, the electronic device can perform quality checks on the second marker set after obtaining it.

[0053] In one example, assume that the second preset number of marker points corresponding to the second marker is set to 8, and the second marker point set is the result of a single frame measurement. Calculate the number of data points in the second marker point set. If the number of data points in the second marker point set is 6, it means that the number of data points in the second marker point set is less than the second preset number. If the number of data points in the second marker point set does not meet the second preset number, the tracker can be used to re-track the scanner.

[0054] In another example, assuming the second marker set is the result of multi-frame measurements, the second marker set can include multiple second subsets, each representing the result of a single-frame measurement of the second marker. If the second marker set is the result of multi-frame measurements, the second preset quantity can be a baseline quantity set for each second subset. For example, the second preset quantity of markers corresponding to the second marker is set to 8, representing the minimum number of markers expected to be measured in a single frame under ideal conditions. The number of data points in each second subset is calculated. If the number of data points in any second subset is 6, it indicates that the number of data points in that second subset is less than the second preset quantity. This determines that the number of data points in that second subset does not meet the second preset quantity, and that second subset can be set as invalid data. Multiple second subsets that meet the second preset quantity are acquired. If the number of multiple second subsets that meet the second preset quantity does not meet the target quantity, the tracker is reused to reacquire the second marker set, i.e., multiple second subsets are reacquired.

[0055] Based on the results of a single frame measurement, if the number of data points in the first set of marker points meets the first preset number, and the number of data points in the second set of marker points meets the second preset number, it is determined that the first set of marker points and the second set of marker points meet the quality requirements, and then step S302 is executed.

[0056] Based on the results of multi-frame measurements, if the first subset of multi-frame measurements satisfies the target quantity for a first preset number, and the second subset of multi-frame measurements satisfies the target quantity for a second preset number, and the marker set and the second marker set meet the quality requirements, then proceed to step S302.

[0057] In other embodiments of this application, the quality of the first and second marker point sets can be verified not only by checking the number of points in the first and second marker point sets, but also by checking their geometric distribution. In one example, the first and second marker point sets should each constitute a non-coplanar, widely distributed, and geometrically well-structured point set in space, avoiding the concentration of all points in a small area or their approximate location on the same line or plane.

[0058] In another example, the quality can also be verified by checking the state of individual data points in the first and second set of marker points. For example, checking whether the reflection intensity of an individual data point is sufficient and uniform, whether the edges are sharp, and whether the image size is stable.

[0059] The above are merely examples; this application does not limit the method for detecting the quality of the first and second marker sets.

[0060] Step S302: Based on the first set of marker points and the second set of marker points, complete the calibration of the target transformation relationship between the local coordinate system corresponding to the first marker and the scanning coordinate system of the scanner.

[0061] In some embodiments of this application, a first reference coordinate set of the first marker in a local coordinate system is obtained. The local coordinate system may be the inherent coordinate system of the first marker itself, that is, the local coordinate system defined and fixed on the structure of the first marker at the time of manufacture.

[0062] Since the second marker (such as a target point) is rigidly fixed to the scanner, the transformation relationship between the inherent coordinate system of the second marker and the scanning coordinate system of the scanner has been determined through the previous calibration and alignment process. Therefore, in the subsequent calibration process, the "inherent coordinate system" of the second marker can be regarded as already aligned with the scanning coordinate system, and the factory-preset coordinate set of the second marker can be directly expressed as the known coordinate set in the scanning coordinate system, that is, the known second reference coordinate set of the second marker in the scanning coordinate system.

[0063] In some embodiments of this application, a first reference coordinate set, a second reference coordinate set, a first marker set, and a second marker set are used to calibrate the target transformation relationship. Specifically, a first transformation relationship is determined based on the first marker set and the first reference coordinate set. This first transformation relationship is the relative positional relationship between the local coordinate system and the tracking coordinate system. This can be expressed by the following formula: ,in, Indicates the first The first transformation relationship corresponding to the frame includes rotation and translation; Indicates the first The first set of markers in the frame measurement Frame 1 subset, number The first subset of the frame includes the set of coordinates of multiple marker points on the first marker in the tracking coordinate system of the tracker; This represents the first reference coordinate set.

[0064] Based on the second set of marker points and the second set of reference coordinates, a second transformation relationship is determined. This second transformation relationship represents the relative positional relationship between the scanning coordinate system and the tracking coordinate system. It can be expressed as a formula: ,in, Indicates the first The second transformation relationship corresponding to the frame includes rotation and translation; Indicates the first The second set of markers in the frame measurement Frame 2 subset, number The second subset of the frame includes the set of coordinates of multiple target points on the second marker in the tracking coordinate system of the tracker; This represents the second reference coordinate set.

[0065] After determining the first and second transformation relationships, the target transformation relationship can be determined based on these relationships. This can be expressed by the formula: ,in, Indicates the target transformation relationship; Indicates the first The inverse matrix corresponding to the second transformation relation of the frame; Indicates the first The first transformation relationship corresponding to the frame.

[0066] In other embodiments of this application, besides being based on a single frame (as described above), In addition to determining the target transformation relationship based on a single frame, to improve calibration accuracy, the target transformation relationship can also be determined based on multiple frames. In one example, the first set of marker points includes a first subset of multiple frames, and the second set of marker points includes a second subset of multiple frames. Then, the target transformation relationship is expressed by the formula: ,in, Determined by the second subset of multiple frames; Determined by the first subset of multiple frames; detailed determination method can be found below. Figure 5 The illustrated embodiment.

[0067] In some embodiments of this application, after the target transformation relationship is determined, the calibration accuracy of the target transformation relationship can be verified.

[0068] In one example, taking the target transformation relationship determined based on a single frame as an example, the target transformation relationship determined in a single frame is recorded as the candidate relationship of the i-th frame. The first set of marker points and the second set of marker points in the tracking coordinate system are obtained. Based on the first set of marker points and the second set of marker points, the third transformation relationship of the first marker and the second marker in the i-th frame is calculated. The first difference between the third transformation relationship of the i-th frame and the candidate relationship of the i-th frame is calculated. If the first difference is less than or equal to a preset difference, it indicates that the candidate relationship of the i-th frame meets the calibration accuracy requirements, and the candidate relationship of the i-th frame is used as the target transformation relationship. If the first difference is greater than the preset difference, it indicates that the candidate relationship of the i-th frame does not meet the calibration accuracy requirements, and the target transformation relationship is recalibrated. The process can then return to step S301.

[0069] In another example, taking the target transformation relationship determined based on multiple frames as an example, the first and second marker point sets in the tracking coordinate system are obtained. Based on the first and second marker point sets, the fourth transformation relationship corresponding to the first and second markers in multiple frames is calculated. The second difference between the fourth transformation relationship and the target transformation relationship is calculated. If the second difference is less than or equal to a preset difference, it indicates that the target transformation relationship meets the calibration accuracy requirements, and the calibration of the transformation relationship between the local coordinate system corresponding to the first marker and the scanner's scanning coordinate system is completed. If the second difference is greater than the preset difference, it indicates that the target transformation relationship does not meet the calibration accuracy requirements, and the target transformation relationship is recalibrated. The process can then return to step S301.

[0070] Through the above embodiments, the scanner is equipped with a first marker and a second marker. The relative position between the first marker and the scanner is fixed, and the second marker is fixed on the scanner. This allows calibration to be completed without adding additional calibration devices (such as hand-eye calibrators), reducing calibration costs. Specifically, a tracker tracks the first and second markers on the scanner to obtain a first set of marker points for the first marker in the tracking coordinate system and a second set of marker points for the second marker in the tracking coordinate system, providing data for subsequent calibration. Therefore, based on the first and second marker point sets, the target transformation relationship between the local coordinate system corresponding to the first marker and the scanner's scanning coordinate system can be calibrated, simultaneously ensuring both calibration accuracy and efficiency.

[0071] In addition, no additional calibration equipment is needed during the calibration process, and additional errors caused by calibration equipment can also be avoided.

[0072] Figure 5 This is a flowchart of a calibration method provided in another embodiment of this application. The first set of marker points may include a multi-frame first subset generated from multiple scans, and the second set of marker points may include a multi-frame second subset generated from multiple scans. The multi-frame first subset and multi-frame second subset can be used to complete the calibration of the target transformation relationship, thereby reducing calibration errors. Figure 5 As shown, it includes the following steps: Step S501: Based on the first subset of multiple frames and the first reference coordinate set, generate the first transformation relationship corresponding to the multiple frames.

[0073] In some embodiments of this application, a first transformation relationship between each frame in the first subset of multiple frames and the first reference coordinate set is calculated.

[0074] In one example, assuming there exists a first subset of K frames, it is calculated using the following formula: ,in, Indicates the first The first transformation relationship corresponding to the frame includes rotation and translation; Indicates the first The first set of marker points in the frame measurement Frame 1 subset, number The first subset of the frame includes the set of coordinates of multiple marker points on the first marker in the tracking coordinate system of the tracker; This represents the first reference coordinate set. Therefore, the first transformation relationship corresponding to K frames is used as the first transformation relationship corresponding to multiple frames.

[0075] Step S502: Based on the second subset of multiple frames and the second reference coordinate set, generate the second transformation relationship corresponding to the multiple frames.

[0076] In some embodiments of this application, a second transformation relationship between each frame in the second subset of multiple frames and the second reference coordinate set is calculated.

[0077] In one example, assuming there exists a second subset of K frames, it is calculated using the following formula: ,in, Indicates the first The second transformation relationship corresponding to the frame includes rotation and translation; Indicates the first The second set of markers in the frame measurement Frame 2 subset, number The second subset of the frame includes the set of coordinates of multiple target points on the second marker in the tracking coordinate system of the tracker; This represents the second reference coordinate set. Therefore, the second transformation relationship corresponding to K frames is used as the second transformation relationship corresponding to multiple frames.

[0078] Step S503: Calculate the optimal solution for the first transformation relationship and the second transformation relationship corresponding to multiple frames using the hand-eye calibration algorithm, and determine the target transformation relationship.

[0079] In some embodiments of this application, the purpose of the hand-eye calibration algorithm is to directly solve for the optimal fixed transformation X (i.e., the target transformation relationship) using data from all frames.

[0080] For each frame of data (Total K frames), the ideal motion relationship can be expressed as: It can be abbreviated as .

[0081] By collecting Frame data, to obtain the corresponding transform pair By jointly optimizing the transformation relationships of all frames using the least squares method (or the direct linear solution method), the optimal solution can be obtained. .

[0082] The above embodiments improve the calibration accuracy of target transformation relationships through joint optimization.

[0083] Figure 6 This is a flowchart of a calibration method provided in another embodiment of this application. The first set of marker points may include a multi-frame first subset generated from multiple scans, and the second set of marker points may include a multi-frame second subset generated from multiple scans. The multi-frame first subset and multi-frame second subset can be used to complete the calibration of the target transformation relationship. Furthermore, to reduce errors, the calibration accuracy can be verified by examining the target transformation relationship generated based on each frame, such as... Figure 6 As shown, the steps include the following.

[0084] Step S601: Based on the first subset of the i-th frame, the second subset of the i-th frame, the first reference coordinate set, and the second reference coordinate set, determine the candidate relationship of the i-th frame.

[0085] In some embodiments of this application, a first transformation relationship for the i-th frame is determined based on a first subset of the i-th frame and a first reference coordinate set. This first transformation relationship is the relative positional relationship between the local coordinate system and the tracking coordinate system corresponding to the i-th frame. This can be expressed by the formula: ,in, Indicates the first The first transformation relationship corresponding to the frame includes rotation and translation; Indicates the first Frame 1 subset, number The first subset of the frame includes the set of coordinates of multiple marker points on the first marker in the tracking coordinate system of the tracker; Let i represent the first reference coordinate set. Here, i is a positive integer.

[0086] Based on the second subset and the second reference coordinate set of the i-th frame, the second transformation relationship of the i-th frame is determined. This second transformation relationship is the relative positional relationship between the scan coordinate system and the tracking coordinate system corresponding to the i-th frame. This can be expressed by the formula: ,in, Indicates the first The second transformation relationship corresponding to the frame includes rotation and translation; Indicates the first Frame 2 subset, number The second subset of the frame includes the set of coordinates of multiple target points on the second marker in the tracking coordinate system of the tracker; This represents the second reference coordinate set.

[0087] Based on the first and second transformation relationships of the i-th frame, the candidate relationships for the i-th frame are determined. This can be expressed by the formula: ,in, This represents the candidate relationship for the i-th frame; Indicates the first The inverse matrix corresponding to the second transformation relationship of the frame; Indicates the first The first conversion relationship of the frame.

[0088] Step S602: Based on the first subset of the i-th frame and the second subset of the i-th frame, determine the third transformation relationship of the i-th frame.

[0089] In some embodiments of this application, the third transformation relationship of the i-th frame is the relative positional relationship between the first marker and the second marker in the i-th frame. The third transformation relationship of the i-th frame is determined based on the first subset and the second subset of the i-th frame, expressed by the formula: ,in, This represents the third transformation relationship for the i-th frame.

[0090] Step S603: Calculate the first difference between the third transformation relation of the i-th frame and the candidate relation of the i-th frame.

[0091] In some embodiments of this application, the purpose of calculating the first difference is to quantitatively evaluate the accuracy of the candidate relationship of the i-th frame relative to the third transformation relationship of the high-precision i-th frame, thereby determining whether it meets the application accuracy requirements and deciding whether recalibration is needed. Specifically, when calculating the first difference, for the translation part, the Euclidean distance between the two translation vectors is directly calculated. For the rotating part, the relative rotation matrix is ​​calculated. And extract its corresponding rotation angle. These differences are used to measure directional deviations. In essence, they quantify the absolute errors of the candidate relation in spatial position and orientation of the i-th frame, respectively. They are not only an objective measure of the overall error of the calibration system, but also directly reflect the magnitude and reliability of the coordinate transformation deviation that the calibration relation may cause in actual use.

[0092] Step S604: Merge multiple candidate relationships corresponding to the first difference that is less than or equal to the preset difference to complete the calibration of the target transformation relationship.

[0093] In some embodiments of this application, if the first difference corresponding to the i-th frame is less than or equal to a preset difference, it means that the first difference corresponding to the i-th frame meets the quality requirements; if the first difference corresponding to the i-th frame is greater than the preset difference, it means that the first difference corresponding to the i-th frame does not meet the quality requirements.

[0094] Assuming there are K frames, filtering is performed by calculating the first difference. After calculating the first difference for all frames, the first differences less than or equal to a preset difference are selected and used as candidate relations that meet the quality requirements. Then, the candidate relations corresponding to the first differences less than or equal to the preset difference are fused. Specifically, for the translation portion corresponding to the candidate relations that meet the quality requirements, the average or median of all translation portions can be calculated; for the rotation portion corresponding to the candidate relations that meet the quality requirements, the rotation portion can be converted into a quaternion.

[0095] After fusing multiple candidate relations that meet the quality requirements, the fused result is used as the target transformation relation to complete the calibration of the target transformation relation.

[0096] The above embodiments can verify the candidate relationship corresponding to each frame, taking into account both the stability and accuracy of the calibration.

[0097] Figure 7 This is a flowchart illustrating the detection process for the calibration accuracy of the target transformation relationship provided in an embodiment of this application. For example... Figure 3 Implementation examples Figure 5 Implementation examples Figure 6 All implementation examples generate target transformation relationships. To further ensure the accuracy of the target transformation relationships, the following can be performed: Figure 3 Implementation examples Figure 5 Implementation examples Figure 6 The target transformation relationship generated by the example is used to evaluate the calibration accuracy, such as Figure 7 As shown, the steps include the following.

[0098] Step S701: Determine the fourth transformation relationship based on the first set of marker points and the second set of marker points.

[0099] In some embodiments of this application, the fourth transformation relationship can be calculated and determined based on at least one frame of data, and is the relative positional relationship between the first marker and the second marker determined based on the first marker set and the second marker set. The fourth transformation relationship is determined according to the first marker set and the second marker set, expressed by the formula: ,in, This represents the fourth transformation relationship determined based on K frames. It is understood that, since the fourth transformation relationship is determined based on single-frame data, it may include the third transformation relationship of the i-th frame from step S602.

[0100] Step S702: Use the fourth transformation relation to evaluate the calibration accuracy of the target transformation relation.

[0101] In some embodiments of this application, the calibration accuracy of the target transformation relation can be evaluated by calculating the second difference between the fourth transformation relation and the target transformation relation.

[0102] Step S703: Determine whether the target transformation relationship meets the preset calibration accuracy.

[0103] In some embodiments of this application, if the second difference is less than or equal to a preset threshold, it is determined that the target transformation relationship meets the preset calibration accuracy, and step S704 is executed. If the second difference is greater than the preset threshold, it is determined that the target transformation relationship does not meet the preset calibration accuracy, and step S705 is executed.

[0104] Step S704: Complete the calibration of the target transformation relationship.

[0105] Step S705: Recalibrate the target transformation relationship.

[0106] The above embodiments can simultaneously achieve both calibration accuracy and calibration efficiency.

[0107] In other embodiments of this application, after verifying the calibration accuracy of the target transformation relationship, a first transformation relationship between the local coordinate system and the tracking coordinate system of the first marker, a second transformation relationship between the scanning coordinate system and the tracking coordinate system, and a target transformation relationship between the local coordinate system and the scanner's scanning coordinate system can be obtained. Therefore, subsequent 3D scanning operations can be performed based on the first transformation relationship, the second transformation relationship, and the target transformation relationship.

[0108] In one example, a second marker is present on the scanner during a 3D scanning operation. A tracker simultaneously tracks both the first marker (e.g., multiple marker discs) and the second marker (target points pasted on the scanner), acquiring a second transformation relationship (i.e., the pose of the second marker) between the scanning and tracking coordinate systems in real time. Then, based directly on this second transformation relationship, the point cloud data obtained from scanning the specified object is transformed from the scanning coordinate system to the tracking coordinate system, thereby completing the stitching of multi-view point clouds. Note: Tracking the pose of the first marker is not mandatory in this process, but it can be used to assist in verification or establish a global reference.

[0109] In another example, during a 3D scanning operation, there is no second marker on the scanner, but a rigid connection is maintained between the first marker (such as multiple marker disks) and the scanner. The tracker tracks the first marker to obtain its pose, thereby acquiring the first transformation relationship between the local coordinate system and the tracking coordinate system in real time. Based on the target transformation relationship and the pose corresponding to the first marker, the pose of the scanner in the tracking coordinate system (i.e., the second transformation relationship) is obtained. Then, the derived second transformation relationship is used to transform the point cloud data from the scanning coordinate system to the tracking coordinate system, completing the stitching.

[0110] Figure 8 This is a schematic diagram of the structure of the electronic device provided in an embodiment of this application. For example... Figure 8 As shown, the electronic device 10 may include a display device 100, a communication module 101, a memory 102, a processor 103, an input / output (I / O) interface 104, and a bus 105. The processor 103 is coupled to the display device 100, the communication module 101, the memory 102, and the I / O interface 104 via the bus 105.

[0111] The display device 100 may be a touch screen, specifically a touch-sensitive liquid crystal display device. Alternatively, the display device 100 may be a non-touch screen. The display device 100 is used to display scan data and calibration results.

[0112] Communication module 101 may include a wired communication module and / or a wireless communication module. The wired communication module may provide one or more wired communication solutions such as Universal Serial Bus (USB) and Controller Area Network (CAN). The wireless communication module may provide one or more wireless communication solutions such as Wireless Fidelity (Wi-Fi), Bluetooth (BT), mobile communication networks, Frequency Modulation (FM), Near Field Communication (NFC), and Infrared (IR). The memory 102 may include one or more random access memory (RAM) and one or more non-volatile memory (NVM). The RAM can be directly read and written by the processor 103, and can be used to store executable programs (such as machine instructions) of the operating system or other running programs, as well as user and application data.

[0113] Random access memory can include static random-access memory (SRAM), dynamic random-access memory (DRAM), synchronous dynamic random-access memory (SDRAM), double data rate synchronous dynamic random-access memory (DDR SDRAM), etc.

[0114] Non-volatile memory can also store executable programs and user and application data, and can be pre-loaded into random access memory for direct reading and writing by the processor 103. Non-volatile memory can include disk storage devices and flash memory.

[0115] Memory 102 is used to store one or more computer programs. The one or more computer programs are configured to be executed by processor 103. The one or more computer programs include multiple instructions that, when executed by processor 103, can implement a calibration method that is executed on electronic device 10.

[0116] In other embodiments, the electronic device 10 also includes an external memory interface for connecting to an external memory to expand the storage capacity of the electronic device 10.

[0117] Processor 103 may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors.

[0118] The processor 103 provides computing and control capabilities; for example, the processor 103 is used to execute computer programs stored in the memory 102 to implement the calibration method described above.

[0119] I / O interface 104 is used to provide a channel for user input or output. For example, I / O interface 104 can be used to connect various input and output devices, such as mouse, keyboard, touch device, display screen, etc., so that users can enter information or visualize information.

[0120] Bus 105 is used at least to provide a channel for communication between communication modules 101, memory 102, processor 103, and I / O interface 104 in electronic device 10.

[0121] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 10. In other embodiments of this application, the electronic device 10 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0122] This application also provides a computer-readable storage medium storing a computer program, which includes program instructions. When the program instructions are executed, the method implemented can refer to the methods in the above embodiments of this application.

[0123] The computer-readable storage medium can be the internal memory of the electronic device described in the above embodiments, such as the hard disk or memory of the electronic device. Alternatively, the computer-readable storage medium can be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., provided on the electronic device.

[0124] In some embodiments, a computer-readable storage medium may include a stored program area and a stored data area, wherein the stored program area may store an operating system, an application program required for at least one function, etc.; and the stored data area may store data created based on the use of the electronic device, etc.

[0125] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0126] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0127] In the embodiments provided in this application, it should be understood that the disclosed apparatus / terminal devices and methods can be implemented in other ways. For example, the apparatus / terminal device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0128] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0129] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A calibration method applied to a tracking scanning system, characterized in that, The tracking scanning system includes a tracker and a scanner, and the method includes: The tracker tracks a first marker and a second marker on the scanner to obtain a first marker point set of the first marker in the tracking coordinate system of the tracker, and a second marker point set of the second marker in the tracking coordinate system; wherein the relative positions of the scanner with the first marker and the second marker are fixed, and the first marker is a rigid frame composed of multiple marker point disks. Obtain the first reference coordinate set of the first marker in the local coordinate system; Obtain the second reference coordinate set of the second marker in the scanning coordinate system; Based on the first set of marker points, the second set of marker points, the first set of reference coordinates, and the second set of reference coordinates, the target transformation relationship between the local coordinate system corresponding to the first marker and the scanning coordinate system of the scanner is calibrated, including: determining a first transformation relationship based on the first set of marker points and the first set of reference coordinates, wherein the first transformation relationship is the relative positional relationship between the local coordinate system and the tracking coordinate system; determining a second transformation relationship based on the second set of marker points and the second set of reference coordinates, wherein the second transformation relationship is the relative positional relationship between the scanning coordinate system and the tracking coordinate system; and determining the target transformation relationship based on the first transformation relationship and the second transformation relationship.

2. The calibration method according to claim 1, characterized in that, The first set of marker points includes a first subset of multiple frames, the second set of marker points includes a second subset of multiple frames, and the method further includes: Based on the first subset of the multiple frames and the first reference coordinate set, a first transformation relationship corresponding to the multiple frames is generated; Based on the second subset of the multiple frames and the second reference coordinate set, a second transformation relationship corresponding to the multiple frames is generated; The target transformation relationship is determined by using a hand-eye calibration algorithm to calculate the optimal solution for the first transformation relationship and the second transformation relationship corresponding to the multiple frames.

3. The calibration method according to claim 1, characterized in that, The first set of marker points includes a first subset of multiple frames, the second set of marker points includes a second subset of multiple frames, and the method further includes: Based on the first subset of the i-th frame, the second subset of the i-th frame, the first reference coordinate set, and the second reference coordinate set, determine the candidate relationship of the i-th frame; i is a positive integer; Based on the first subset and the second subset of the i-th frame, a third transformation relationship of the i-th frame is determined; the third transformation relationship of the i-th frame is the relative positional relationship between the first marker and the second marker in the i-th frame. Calculate the first difference between the third transformation relationship of the i-th frame and the candidate relationship of the i-th frame; By merging multiple candidate relationships corresponding to a first difference that is less than or equal to a preset difference, the calibration of the target transformation relationship is completed.

4. The calibration method according to any one of claims 1 to 3, characterized in that, Based on the first set of marker points and the second set of marker points, the target transformation relationship between the local coordinate system corresponding to the first marker and the scanning coordinate system of the scanner is calibrated, including: Based on the first set of markers and the second set of markers, a fourth transformation relationship is determined, wherein the fourth transformation relationship is the relative positional relationship between the first marker and the second marker determined based on the first set of markers and the second set of markers; The calibration accuracy of the target transformation relationship is evaluated using the fourth transformation relationship. If the target transformation relationship meets the preset calibration accuracy, then the calibration of the target transformation relationship is completed; If the target transformation relationship does not meet the preset calibration accuracy, then the target transformation relationship is recalibrated.

5. The calibration method according to claim 4, characterized in that, The method further includes: Calculate the second difference between the fourth transformation relation and the target transformation relation; If the second difference is less than or equal to a preset threshold, it is determined that the target transformation relationship meets the preset calibration accuracy, and the calibration of the target transformation relationship is completed; If the second difference is greater than a preset threshold, it is determined that the target transformation relationship does not meet the preset calibration accuracy, and then the target transformation relationship is recalibrated.

6. The calibration method according to claim 1, characterized in that, The method further includes: Obtain the data points in the first set of markers and the data points in the second set of markers; If the number of data points in the first set of markers does not meet the first preset number, and / or the number of data points in the second set of markers does not meet the second preset number, then the tracker is used again to track the scanner. If the number of data points in the first set of markers meets a first preset number, and the number of data points in the second set of markers meets a second preset number, then the step of calibrating the target transformation relationship between the local coordinate system corresponding to the first marker and the scanning coordinate system of the scanner based on the first set of markers and the second set of markers is executed.

7. The calibration method according to claim 1, characterized in that, After calibrating the target transformation relationship between the local coordinate system corresponding to the first marker and the scanning coordinate system of the scanner, the method further includes: The specified object is scanned using the scanner to obtain point cloud data of the surface of the specified object; The first marker is tracked using the tracker to obtain the pose of the first marker. Based on the pose of the first marker and the transformation relationship with the target, the point cloud data is transformed into the tracking coordinate system.

8. A tracking scanning system for implementing the calibration method as described in any one of claims 1 to 7, characterized in that, The tracking scanning system includes: A scanner is used to carry a first marker and a second marker during the calibration process, and the relative positions of the scanner with respect to the first marker and the second marker are fixed respectively. A tracker is used to track the first marker and the second marker, thereby measuring a first set of marker points for the first marker and a second set of marker points for the second marker; An electronic device is used to determine the target transformation relationship between the local coordinate system corresponding to the first marker and the scanning coordinate system of the scanner, based on the first set of marker points and the second set of marker points.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one instruction that, when executed by a processor, implements the calibration method as described in any one of claims 1 to 7.

Images