Scanner calibration method and device, electronic equipment and medium

Abstract

The embodiment of the invention relates to a scanner calibration method and device, electronic equipment and a medium, and relates to the technical field of three-dimensional scanning, a scanner is detachably connected to a tracking frame, the scanner comprises a shell and a first multi-view camera located on the first surface of the shell, and the tracking frame comprises a main body frame and an extension frame. The extension frame comprises a plurality of first marks, when the scanner is connected to the main body frame, the first multi-view camera faces the outside of the main body frame and the field of view of the first multi-view camera covers at least part of the first marks, and at least part of the first marks of the extension frame are captured by using the first multi-view camera; multiple frames of to-be-processed images including at least part of the first marks are obtained; and processing based on the multiple frames of to-be-processed images to obtain a relative position relationship between the scanner and the tracking frame. By adopting the technical scheme, both tracking scanning and handheld scanning in a narrow area can be considered, and the scanner can be calibrated again only by means of the tracking frame after resetting.

Description

Technical Field

[0001] This disclosure relates to the field of 3D scanner technology, and in particular to a scanner calibration method, apparatus, electronic device, and medium. Background Technology

[0002] In the field of 3D scanning, optical trackers can be used in conjunction with scanners with tracking frames to complete non-patch tracking. Although scanners with tracking frames can be used as scanners on their own, for example, to scan some patch objects or to complete scanning using texture stitching, the tracking frames are relatively large, making it impossible to complete scanning in many narrow spaces. Summary of the Invention

[0003] In order to solve the above-mentioned technical problems, or at least partially solve the above-mentioned technical problems, this disclosure provides a scanner calibration method, apparatus, electronic device and medium.

[0004] This disclosure provides a scanner calibration method. The scanner is detachably connected to a tracking frame. The scanner includes a housing and a first multi-view camera located on a first surface of the housing. The tracking frame includes a main frame and an extension frame. The extension frame includes a plurality of first marks. When the scanner is connected to the main frame, the first multi-view camera faces outward from the main frame and the field of view of the first multi-view camera covers at least a portion of the first marks. The method includes: using the first multi-view camera to capture the at least a portion of the first marks on the extension frame to obtain a multi-frame image to be processed including the at least a portion of the first marks; and processing the multi-frame image to obtain the relative positional relationship between the scanner and the tracking frame.

[0005] This disclosure also provides a scanner calibration device, wherein the scanner is detachably connected to a tracking frame, the scanner includes a housing and a first multi-view camera located on a first surface of the housing, the tracking frame includes a main frame and an extension frame, the extension frame includes a plurality of first marks, when the scanner is connected to the main frame, the first multi-view camera faces outward from the main frame and the field of view of the first multi-view camera covers at least a portion of the first marks, the device includes: a capture module for capturing the at least a portion of the first marks of the extension frame using the first multi-view camera to obtain a multi-frame image to be processed including the at least a portion of the first marks; and a processing module for processing the multi-frame image to obtain the relative positional relationship between the scanner and the tracking frame.

[0006] This disclosure also provides an electronic device, the electronic device comprising: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory and execute the instructions to implement the scanner calibration method provided in this disclosure.

[0007] This disclosure also provides a computer-readable storage medium storing a computer program for performing the scanner calibration method provided in this disclosure.

[0008] This disclosure also provides a computer program product, including a computer program, wherein the computer program is executed by a processor as described in this disclosure regarding the scanner calibration method.

[0009] Compared with the prior art, the technical solution provided in this disclosure has the following advantages: The scanner calibration scheme provided in this disclosure allows for detachable connection of the scanner to a tracking frame. The scanner includes a housing and a first multi-view camera located on a first surface of the housing. The tracking frame includes a main frame and an extension frame. The extension frame includes multiple first marks. When the scanner is connected to the main frame, the first multi-view camera faces outward from the main frame, and its field of view covers at least a portion of the first marks. By capturing at least a portion of the first marks on the extension frame using the first multi-view camera, multiple frames of images to be processed, including at least a portion of the first marks, are obtained. Based on these multiple frames, the relative positional relationship between the scanner and the tracking frame is obtained. Using the above technical solution, the scanner with a detachable tracking frame can handle both tracking scanning and handheld scanning in confined areas. After disassembly and reinstallation, it solves the technical problem of changes in the relative positional relationship between the scanner and the tracking frame due to possible impacts, friction, or errors (such as travel errors of the detachable structure). After resetting, the scanner can be recalibrated using only the tracking frame itself, without the need for dedicated calibration plates or calibration objects. This further improves the scanner calibration efficiency while ensuring the calibration effect. Attached Figure Description

[0010] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0011] Figure 1A This is an application scenario diagram of a scanner calibration method provided in an embodiment of the present disclosure; Figure 1B An application scenario diagram of another scanner calibration method provided in this disclosure embodiment; Figure 1C This is a schematic diagram of the scanner provided in an embodiment of the present disclosure; Figure 2 A schematic flowchart illustrating a scanner calibration method provided in this embodiment of the disclosure; Figure 3 A schematic flowchart illustrating another scanner calibration method provided in this embodiment of the disclosure; Figure 4 This is a schematic diagram of the structure of a scanner calibration device provided in an embodiment of the present disclosure; Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0012] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0013] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0014] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0015] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0016] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0017] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0018] In the field of 3D scanning, optical trackers can be used in conjunction with scanners with tracking frames to achieve non-stick tracking. Although scanners with tracking frames can be used as scanners on their own, the tracking frames are relatively large, making it impossible to scan in many confined spaces. In addition, due to the presence of the tracking frames, they are also heavier than ordinary handheld scanners.

[0019] To address the aforementioned problems, this disclosure proposes a scanner calibration method. The scanner is detachably connected to a tracking frame. The scanner includes a housing and a first multi-view camera located on a first surface of the housing. The tracking frame includes a main frame and an extension frame. The extension frame includes multiple first marks. When the scanner is connected to the main frame, the first multi-view camera faces outward from the main frame, and its field of view covers at least a portion of the first marks. By capturing at least a portion of the first marks on the extension frame using the first multi-view camera, multiple frames of images to be processed, including at least a portion of the first marks, are obtained. Processing is performed based on these multiple frames to obtain the relative positional relationship between the scanner and the tracking frame. Using this technical solution, a scanner with a detachable tracking frame can select either tracking scanning or handheld scanning in confined areas as needed. After disassembly and reinstallation, the technical problem of changes in the relative positional relationship between the scanner and the tracking frame due to possible impacts, friction, or errors (such as travel errors of the detachable structure) is solved. After resetting, the scanner can be recalibrated using only the tracking frame itself, without the need for dedicated calibration plates or calibration objects, thus improving scanner calibration efficiency while ensuring the calibration effect.

[0020] The scanner calibration method of this application can be applied to, for example... Figure 1A In the application environment shown, scanner 1 is detachably connected to tracking frame 2. Scanner 1 includes a housing and a first multi-view camera located on a first surface of the housing. Tracking frame 2 includes a main frame 21, which includes branches 211 and joints 212. Extension branches 213 can be provided on the branches 211, for example... Figure 1A The diagram shows two extension branches 213 serving as an extension frame. This extension frame can be foldable, flexible, or detachable, allowing it to be retracted after calibration. This ensures scanner calibration efficiency without affecting the actual scanning process. The extension frame includes multiple first marks 2131. Figure 1A The multiple first marks 2131 shown are circular marking points. The form of the first marks 2131 is not a limitation of this disclosure; for example, they can be triangles, quadrilaterals, or patterns such as QR codes. When the scanner 1 is connected to the main frame 21, the first multi-view camera faces outward from the main frame 21, and the field of view of the first multi-view camera covers at least part of the first marks. Furthermore, Figure 1AIt was also shown that the tracking framework 2 may include a target 22.

[0021] It is understood that this document does not limit the number and form of the extension branches 213. For example, the vertices of the extension branches 213 may include a stereo target, and the stereo target may have multiple first marks 2131. This document also does not limit the number and form of the first marks 2131, as long as it ensures that when the scanner 1 is connected to the main frame 21, the field of view of the first multi-view camera can cover at least part of the first marks 2131 to complete the calibration of the relative positional relationship. Specifically, the scanner 1 is detachably connected to the tracking frame 2, for example, as shown in the example. Figure 1B As shown, the scanner 1, tracking frame 2, and handle 3 are included. The tracking frame 2 includes a space 4 for detachably connecting the scanner 1. Figure 1B The scanner 1 and handle 3 shown are connected to the tracking frame 2 to form the configuration described above. Figure 1A The scanner shown has a tracking frame. It can be understood that in some examples, the handle 3 can be replaced by part of the main structure of the tracking frame 2 instead of being a separate component.

[0022] Specifically, Figure 1C An exemplary structural schematic diagram of scanner 1 is shown. In the depicted embodiment, scanner 1 includes a frame structure 11 and an imaging module 12 located on the frame structure 11. The frame structure 11 can describe the housing included in the scanner according to embodiments of this disclosure, and the imaging modules 12 can be arranged side by side such that the fields of view of each imaging module at least partially overlap. In some embodiments, the imaging module 12 may include three cameras, namely a first camera 121, a second camera 122, and a third camera 124, for example, combining the first camera 121 and the third camera 124 into a binocular camera as a first multi-view camera. It is understood that... Figure 1B The scanner in the image module 12 has a surface as its first surface. The imaging module 12 may also include a light projector 123, which may include a light source, a collimating lens, and diffractive optical elements. The light source is configured to emit a light beam towards the lens. The light beam, collimated by the collimating lens, propagates to the diffractive optical elements, which replicate the focused light beam to form a speckle pattern or fringe pattern and project it onto the scanned object / area. The speckle image or fringe pattern is reflected back from the scanned object / area and acquired by the imaging module 12 for further processing by a processing chip (not shown) to obtain three-dimensional information of the scanned object / area. In other embodiments, the light projector 123 may be an image projector, such as a digital micromirror device, a liquid crystal display projector, or an organic electroluminescent display projector.

[0023] In some embodiments, the light projector 123 may include a single light source, such as a light source emitting infrared light, white light, blue light, or other visible monochromatic light. In other embodiments, the light projector 123 is configured to emit light with wavelengths between 405 nm and 1100 nm. In still other embodiments, the light projector 123 may include two or three identical light sources, such as two or three light sources emitting infrared light. Alternatively, the light projector 123 may include two or three different light sources, such as a first light source emitting infrared light and a second light source emitting white light, or a first light source emitting infrared light, a second light source emitting white light, and a third light source emitting blue light. The two or three identical light sources may be part of the same light projector 123 or may be implemented as separate units (e.g., in an additional light projector unit), and similarly, the two or three different light sources may be part of the same light projector 123 or may be implemented as separate units (e.g., in an additional light projector unit).

[0024] In some embodiments, the imaging module 12 may also include another light projector (not shown), such as a speckle pattern projector, a stripe pattern projector, or an image projector.

[0025] The first camera 121 and the third camera 124 are typically monochrome (e.g., black and white) cameras, and will depend on the type of light source(s) used in the light projector(s) 123. In some embodiments, the first camera 121 and the third camera 124 may be monochrome, visible spectrum, or near-infrared cameras, and the light projector 123 may be an infrared or near-infrared light projector. The first camera 121 and the third camera 124 may use any suitable shutter technology, including but not limited to rolling shutters, global shutters, mechanical shutters, and optical liquid crystal display (LCD) shutters. In some embodiments, the second camera 122 may be a color camera (also referred to as a texture camera). The texture camera may use any suitable shutter technology, including but not limited to rolling shutters, global shutters, mechanical shutters, and optical liquid crystal display (LCD) shutters. In some embodiments, the first camera 121, the second camera 122, and the third camera 124 may have similar configurations to improve matching confidence and speed. In other embodiments, the imaging module 12 may also include a fourth camera, such that the scanner includes three monochrome cameras and one color camera. In a further embodiment, the imaging module can also use a single camera to capture reflected light and color textures, omitting the second (and third and / or fourth) camera. In other embodiments, the imaging module 12 may also include a fourth and a fifth camera, such that the scanner includes four monochrome cameras and one color camera. For example, a first multi-view camera can be formed using a first camera 121 and a third camera 124, and a second multi-view camera can be formed using a fourth and a fifth camera. The first and second multi-view cameras may have different focal lengths and / or different field of view angles.

[0026] Continue to refer to, for example Figure 1C As shown, the first camera 121, the second camera 122, the light projector 123, and the third camera 124 can be located side-by-side on a surface of the frame structure 12, such as the first surface described herein. A surface located on the frame structure 12 and opposite to the first surface can be referred to as the second surface. The first camera 121, the second camera 122, the light projector 123, and the third camera 124 are spaced apart from each other and all face directly in front of a surface. In some examples, the first camera 121 has a first field of view facing a region in front of a surface, the second camera 122 has a second field of view facing a region in front of a surface, the third camera 124 has a third field of view facing a region in front of a surface, and the light projector 123 has a projected field of view facing a region in front of a surface. In some examples, the first field of view and the projected field of view at least partially overlap, the second field of view and the projected field of view at least partially overlap, the third field of view and the projected field of view at least partially overlap, and the first field of view, the second field of view, and the third field of view at least partially overlap.

[0027] A data connection (such as USB, serial communication connection) between scanner 1 and one or more computer processors (not shown) allows the transmission of data collected by the first camera 121, the second camera 122, and the third camera 124, enabling it to be processed to derive 3D measurements of the scanned object / surface. The one or more computer processors may be implemented in a remote computing system (electronic device), or alternatively, may be part of scanner 1 itself.

[0028] For example, light projector 123 includes a single light projector unit, or it may have two or more light projector units. The light projector unit can be configured to project visible or invisible light, coherent or incoherent light. In some embodiments, the light projector unit may include one or more light sources consisting of lasers (e.g., vertical cavity surface emitter (VCSEL), edge emitter (EEL), solid-state lasers, semiconductor lasers, and / or one or more LEDs (or OLEDs)).

[0029] A light projector unit can be configured to project a structured light pattern consisting of multiple light sheets arranged side-by-side. When the light sheets are projected onto the surface of an object, they can appear as elongated light stripes. These elongated light stripes are non-intersecting and, in some embodiments, can be substantially parallel to each other, while in others, they can intersect each other. In some embodiments, the light sheets can also appear as dense dots or spots, such as a collection of dots or spots of different sizes. In some embodiments, the light projector unit can be a programmable light projector unit capable of projecting more than one light pattern. For example, the light projector unit can be configured to project different structured line patterns. In some embodiments, the light projector unit can emit light with wavelengths between 405 nm and 1100 nm.

[0030] In some examples, using a first camera 121 and a third camera 124, two images of an object can be captured simultaneously. Image processing can be applied, for example, to computational methods implemented by one or more processors, or to computational methods implemented, for example, by electronic devices, to derive 3D measurements of the surface of the scanned object / object.

[0031] In some examples, the second camera 122 can capture the texture of the object while the first camera 121 and the second camera 124 are capturing images of the object, and the texture can be applied to a computational method, for example, implemented by one or more processors, or to a computational method, for example, implemented by an electronic device, to map onto a 3D measurement of the surface of the scanned object / object.

[0032] In some examples, using a membrane / film with bandpass filter functionality fixed to the lens of a camera (e.g., first camera 121, second camera 122, third camera 124) can match the wavelength of the projector units (multiple), which can help reduce light source interference from ambient light and other projector units.

[0033] In some examples, a calibration plate, such as a single plate or a set of plates whose true geometric distance values ​​have been measured in advance using high-precision methods such as photogrammetry, is used to measure the intrinsic and extrinsic parameters of the first camera 121, the second camera 122, and the third camera 124. The measurement process typically involves a series of consecutive image acquisitions using the scanner 1 after adjusting the calibration plate to different positions, and the calculation of the spatial position and orientation of the first camera 121, the second camera 122, and the third camera 124 by identifying the positions of reference (marked) points / regions / lines in the calibration images, thereby completing the calibration of the intrinsic and extrinsic parameters of the first camera 121, the second camera 122, and the third camera 124.

[0034] Specifically, Figure 2 This is a flowchart illustrating a scanner calibration method provided in an embodiment of the present disclosure. The method can be executed by a scanner calibration device, which can be implemented using software and / or hardware, and is generally integrated into an electronic device. Figure 2 As shown, the method includes: Step 201: Use a first multi-view camera to capture at least a portion of the first marker of the extended frame to obtain a multi-frame image to be processed, including at least a portion of the first marker.

[0035] Specifically, the scanner is detachably connected to the tracking frame; wherein the tracking frame can be a spherical frame, a square frame, or other regular or irregular shaped frame, and the form of the tracking frame is not a limitation of this disclosure. The scanner includes a housing and a first multi-view camera located on a first surface of the housing, the tracking frame includes a main frame and an extension frame, the extension frame includes a plurality of first marks, and when the scanner is connected to the main frame, the first multi-view camera faces outward from the main frame and the field of view of the first multi-view camera covers at least a portion of the first marks.

[0036] The scanner features a detachable tracking frame, which can be locked and disassembled using switches, latches, slots, or any other suitable mechanism. This allows for both tracking-based scanning and standard standalone scanning. The detached scanner, due to its reduced size, can scan confined spaces, expanding its applicability. The detachable tracking frame also enables the scanner, which is paired with the optical tracker, to function as a standalone handheld scanner for scanning confined spaces, further enhancing its flexibility and meeting user needs.

[0037] In this embodiment, because the tracking frame and scanner are detachable, when the disassembled scanner is reinstalled on the tracking frame for reset, the previously calibrated relative positional relationship between the scanner and the tracking frame may change, for example, due to travel errors in the mechanical structure. When the tracking device is used to track the tracking frame again, tracking fails because the changed relative positional relationship is not updated. Therefore, it is necessary to calibrate the new hand-eye relationship (tracking frame and scanner).

[0038] The first multi-view camera is a multi-view camera, such as a binocular camera, disposed on the first surface of the upper housing of the scanner, for capturing at least a portion of the first mark of the extended frame to obtain multiple frames of images to be processed, including at least a portion of the first mark.

[0039] The extended frame is equipped with multiple first markers; the first markers can identify the corresponding positional features on the extended frame, such as using coded patterns, QR codes, one-dimensional codes, stripes, or markers with single-sided or multi-sided surface areas, such as circles (ovals), triangles, squares, rectangles, or combinations of polygons with different numbers of vertices, for example, configured as circular marker points.

[0040] In the embodiments of this disclosure, there are many ways to use a first multi-view camera to capture at least a portion of the first markers of the extended frame to obtain multiple frames of images to be processed that include at least a portion of the first markers. In some embodiments, the first multi-view camera is controlled to scan multiple first markers of the tracking frame at a preset frame rate to obtain multiple frames of images to be processed that include a portion of the first markers. In other embodiments, the first multi-view camera is controlled to scan multiple first markers of the tracking frame at a preset frame rate to obtain multiple frames of images to be processed that include all the first markers. The above are merely examples of using a first multi-view camera to capture at least a portion of the first markers of the extended frame to obtain multiple frames of images to be processed that include at least a portion of the first markers.

[0041] Step 202: Process the multiple frames of images to be processed to obtain the relative positional relationship between the scanner and the tracking frame.

[0042] In this embodiment of the disclosure, the method of processing multiple frames of images to obtain the relative positional relationship between the scanner and the tracking frame is selected according to different scenarios. Specifically, by processing multiple frames of images to obtain multiple first three-dimensional coordinate points reconstructed from at least some of the first markers and the real coordinate values ​​of at least some of the first markers, the first rigid body transformation relationship between the scanner and the tracking frame is determined as the relative positional relationship.

[0043] Specifically, after acquiring multiple frames of images to be processed, the system processes these images to obtain the relative positional relationship between the scanner and the tracking frame, enabling the calibration of the scanner and tracking frame after reset.

[0044] The scanner calibration scheme provided in this disclosure includes a scanner detachably connected to a tracking frame. The scanner includes a housing and a first multi-view camera located on a first surface of the housing. The tracking frame includes a main frame and an extension frame. The extension frame includes a plurality of first marks. When the scanner is connected to the main frame, the first multi-view camera faces outward from the main frame, and the field of view of the first multi-view camera covers at least a portion of the first marks. By capturing at least a portion of the first marks on the extension frame using the first multi-view camera, multiple frames of images to be processed, including at least a portion of the first marks, are obtained. Based on the multiple frames of images to be processed, the relative positional relationship between the scanner and the tracking frame is obtained. Using the above technical solution, a scanner with a detachable tracking frame can select either tracking scanning or handheld scanning in confined areas as needed. After disassembly and reinstallation, it solves the technical problem of changes in the relative positional relationship between the scanner and the tracking frame due to possible impacts, friction, or errors (such as travel errors of the detachable structure). After resetting, the scanner can be recalibrated using only the tracking frame itself, without the need for, for example, a dedicated calibration plate or calibration object. This further improves the scanner calibration efficiency while ensuring the scanner calibration effect.

[0045] Based on the description of the above embodiments, since the tracking frame and scanner are detachable, when the disassembled scanner is reinstalled on the tracking frame for reset, due to the travel error of the mechanical structure, it is necessary to calibrate the new hand-eye relationship (tracking frame and scanner). A corresponding calibration method can be selected, which will be discussed below. Figure 3 Provide a detailed description.

[0046] Specifically, Figure 3 A flowchart illustrating another scanner calibration method provided in this disclosure embodiment is shown below. Figure 3 As shown, the method includes: Step 301: Use a first multi-view camera to capture at least a portion of the first marker of the extended frame to obtain a multi-frame image to be processed, including at least a portion of the first marker.

[0047] Step 302: Process based on multiple frames of images to be processed to obtain multiple first three-dimensional coordinate points reconstructed from at least some of the first markers. Calculate based on the multiple first three-dimensional coordinate points and the real coordinate values ​​of at least some of the first markers to determine the first rigid body transformation relationship between the scanner and the tracking frame as the relative position relationship.

[0048] In this embodiment of the disclosure, the first multi-view camera of the scanner, such as a pair of monocular cameras with fixed positions and / or focal lengths, captures multiple first marks arranged on the extension frame of the first multi-view relative tracking frame to obtain multiple frames of images to be processed, including at least some of the first marks.

[0049] Specifically, processing is performed on multiple frames of images to be processed to obtain at least some of the first three-dimensional coordinate points reconstructed from the first markers. This can be understood as obtaining at least some of the two-dimensional coordinate points of the first markers in each frame of the images to be processed, reconstructing based on the relative positions between the first multi-view cameras and the two-dimensional coordinate points to obtain the three-dimensional coordinate points of at least some of the first markers in each frame of the images to be processed, and stitching based on the common first markers in each frame of the images to be processed to obtain multiple first three-dimensional coordinate points in the same coordinate system.

[0050] It is understood that the first marker has a corresponding real coordinate value; further, based on multiple first three-dimensional coordinate points and at least some of the real coordinate values ​​of the first marker, the first rigid body transformation relationship between the scanner and the tracking frame is obtained as the relative position relationship.

[0051] Specifically, calibration can be performed using only the handheld scanner's own first multi-view camera. By extending a portion of the frame points beyond the tracking frame device, i.e. setting an extended frame, the extended frame can be observed by the scanner's first multi-view camera. When the tracking frame is reset onto the scanner, the scanner's first multi-view camera observes multiple first marks on the extended frame, thereby obtaining the rigid body transformation relationship of the scanner relative to the multiple first marks on the extended frame, and thus obtaining the relative positional relationship between the reset tracking frame and the scanner.

[0052] The scanner calibration scheme provided in this disclosure uses a first multi-view camera to capture at least a portion of the first markers on an extended frame, obtaining multiple frames of images to be processed, including at least a portion of the first markers. Based on the multiple frames of images, processing is performed to obtain multiple first three-dimensional coordinate points reconstructed from the at least a portion of the first markers. Based on the multiple first three-dimensional coordinate points and the actual coordinate values ​​of the at least a portion of the first markers, calculations are performed to determine the first rigid body transformation relationship between the scanner and the tracking frame as their relative positional relationship. Therefore, the scanner with a detachable tracking frame can handle both tracking scanning and handheld scanning in confined areas. By processing multiple frames of images to obtain multiple first three-dimensional coordinate points reconstructed from the at least a portion of the first markers and the actual coordinate values ​​of the at least a portion of the first markers, calculations are performed to determine the first rigid body transformation relationship between the scanner and the tracking frame as their relative positional relationship. This solves the technical problem of changes in the relative positional relationship between the scanner and the tracking frame due to possible impacts, friction, or errors. After resetting, the scanner can be recalibrated using only the tracking frame itself, without the need for dedicated calibration plates or calibration objects, thus improving scanner calibration efficiency while ensuring the effectiveness of the calibration.

[0053] In some embodiments, Figure 1A In the application environment shown, the main frame 21 includes a main structure and multiple targets 22. The multiple targets 22 are located on the outside of multiple joints 212 of the main structure. Each target 22 includes multiple second marks. The multiple second marks have real coordinate values. The position of the tracking frame during movement can be determined by tracking at least some of the second marks of the multiple targets using a tracking device. The position of the scanner during movement can be determined based on the position of the tracking frame and the relative positional relationship determined in the foregoing embodiments.

[0054] In this embodiment of the disclosure, the second mark can identify the corresponding positional features on the extended frame, such as using an coded pattern, QR code, barcode, stripe, or a mark with a single-sided or multi-sided surface area, such as a circle (ellipse), triangle, square, rectangle, or a combination of polygons with different numbers of vertices. In some examples, the second mark is configured as a circular mark point.

[0055] Specifically, at least some of the second markers of multiple targets are acquired using a tracking device, such as an optical tracker. The position of the tracking frame during movement is determined based on the true coordinate values ​​of these second markers. Finally, the position of the scanner during movement is determined based on the position of the tracking frame and the calculated relative positional relationship. Thus, by determining the position of the tracking frame during movement using the tracking device and combining it with the relative positional relationship obtained in the aforementioned embodiments, the position of the scanner during movement can be determined. This allows for real-time acquisition of the scanner's position, further ensuring the accuracy of scanner control and meeting the needs of different usage scenarios.

[0056] In some embodiments, the aforementioned scanner calibration method may also monitor the scanner's scanning space and generate a prompt to remove the tracking frame in response to a mismatch between the scanning space and the tracking frame.

[0057] Specifically, the scanner's scanning space can be monitored. For example, the scanner can acquire the scanning object in real time, such as workpieces, artworks, molds, vehicles, etc. Different scanning objects have different scanning spaces. When the scanning space and the tracking frame are incompatible, indicating that the scanning space is too small—for example, when scanning a large workpiece—the tracking frame needs to be removed to scan the interior of the workpiece separately. In response to this mismatch, a prompt to remove the tracking frame is generated, allowing the scanner to be used independently. This enables handheld scanning of confined areas, further improving the scanner's flexibility and meeting the needs of different scenarios.

[0058] In this embodiment, after removing the tracking frame, a prompt to connect the tracking frame is generated in response to the matching of the scanning space and the tracking frame. Specifically, when the scanning space and the tracking frame are matched, it indicates that the scanner needs to be connected to the tracking frame for tracking scanning. Therefore, in response to the matching of the scanning space and the tracking frame, a prompt to connect the tracking frame is generated, and the tracking frame is connected to the scanner based on the matching of the scanning space and the tracking frame, so that the scanner and the tracking frame can be combined for scanning. This achieves tracking scanning, further improving the flexibility of scanner use and meeting the needs of scanner use in different scenarios.

[0059] It should be noted that after each connection to the tracking frame, the relative positional relationship between the scanner and the tracking frame is re-acquired to further ensure the scanning effect of the scanner.

[0060] Understandably, when using a tracking framework, it is not necessary to label the object under test. Similarly, without a tracking framework, it is also possible to avoid labeling and instead use features such as texture and geometry to stitch the data together. Alternatively, the object under test can be labeled and stitched together using these labels, further improving the flexibility of scanning, meeting users' scanning needs for different scenarios, and enhancing the user experience.

[0061] In some embodiments, the scanner further includes one or more light projectors located on the first surface of the housing, through which structured light is projected onto the surface of the object to be measured, an image of the surface is acquired using a first multi-view camera, and the image of the surface is used to generate a three-dimensional shape of the surface.

[0062] In this embodiment of the disclosure, the light projector can be an image projector, such as a digital micromirror device, a liquid crystal display projector, or an organic electroluminescent display projector. The object to be tested can be a workpiece, a person, or a car. For example, structured light is projected onto the surface of the object to be tested through one or more light projectors. The surface image (such as a speckle image or a stripe pattern) is obtained by a first multi-view camera after being reflected back from the surface of the object to be tested. The surface image is then further processed to obtain the three-dimensional shape of the object to be tested.

[0063] Therefore, it is not necessary to affix labels to the object under test. By projecting structured light onto the surface of the object under test and acquiring images of the surface, the three-dimensional shape of the surface of the object under test can be generated. Features such as texture and geometry can be used for stitching, which improves scanning efficiency while ensuring scanning effect.

[0064] In some embodiments, the scanner further includes one or more light projectors and a second multi-view camera located on the first surface of the housing, projecting structured light onto the surface of the object to be measured through the one or more light projectors, acquiring an image of the surface using the second multi-view camera, and generating a three-dimensional shape of the surface using the image of the surface.

[0065] In this embodiment of the disclosure, the second multi-view camera can be located on the second surface of the housing. For example, the surface opposite to the first surface can be used as the second surface. For example, structured light is projected onto the surface of the object under test by one or more light projectors. The surface image (such as speckle image or stripe pattern) is obtained by the second multi-view camera after being reflected back from the surface of the object under test. The surface image is then further processed to obtain the three-dimensional shape of the object under test.

[0066] Therefore, it is not necessary to affix labels to the object under test. By projecting structured light onto the surface of the object under test and acquiring images of the surface, the three-dimensional shape of the surface of the object under test can be generated. Features such as texture and geometry can be used for stitching, which improves scanning efficiency while ensuring scanning effect.

[0067] Figure 4 This is a schematic diagram of a scanner calibration device provided in an embodiment of the present disclosure. The device can be implemented by software and / or hardware and is generally integrated into an electronic device. Figure 4 As shown, a scanner is detachably connected to a tracking frame. The scanner includes a housing and a first multi-view camera located on a first surface of the housing. The tracking frame includes a main frame and an extension frame. The extension frame includes a plurality of first marks. When the scanner is connected to the main frame, the first multi-view camera faces outward from the main frame and the field of view of the first multi-view camera covers at least a portion of the first marks. The device includes: The capture module 401 is used to capture at least a portion of the first marker of the extended frame using the first multi-view camera, and obtain a multi-frame image to be processed including the at least a portion of the first marker; The processing module 402 is used to process the multiple frames of images to be processed to obtain the relative positional relationship between the scanner and the tracking frame.

[0068] Optionally, the plurality of first markers have real coordinate values. The processing module 302 includes: a processing unit, used to process the multiple frames of images to be processed to obtain a plurality of first three-dimensional coordinate points reconstructed from at least some of the first markers; and a calculation unit, used to calculate based on the plurality of first three-dimensional coordinate points and the real coordinate values ​​of at least some of the first markers to determine a first rigid body transformation relationship between the scanner and the tracking frame as the relative position relationship.

[0069] Optionally, the processing unit is specifically configured to: acquire two-dimensional coordinate points of at least a portion of the first markers in each frame of the image to be processed; reconstruct the image based on the relative positions between the first multi-view cameras and the two-dimensional coordinate points to obtain three-dimensional coordinate points of at least a portion of the first markers in each frame of the image to be processed; and stitch together the images based on the common first markers in each frame of the image to be processed to obtain the plurality of first three-dimensional coordinate points in the same coordinate system.

[0070] Optionally, the main frame includes a main structure and multiple targets, the multiple targets being located on the outside of multiple joints of the main structure, each target including multiple second marks having real coordinate values, the device further including: a tracking module for tracking at least a portion of the second marks of the multiple targets using a tracking device to determine the position of the tracking frame during movement; and a determining module for determining the position of the scanner during movement based on the position of the tracking frame and the relative positional relationship.

[0071] Optionally, the device further includes: a monitoring response generation module for monitoring the scanning space of the scanner and generating a prompt to remove the tracking frame in response to the scanning space being incompatible with the tracking frame.

[0072] Optionally, the apparatus further includes a response generation module for generating a prompt to connect to the tracking frame in response to the adaptation of the scan space to the tracking frame.

[0073] Optionally, a reacquisition module is used to reacquire the relative positional relationship between the scanner and the tracking frame after each connection to the tracking frame.

[0074] Optionally, the first multi-view camera is a binocular camera.

[0075] Optionally, the scanner further includes one or more light projectors located on the first surface of the housing, and the device further includes: a first projection module for projecting structured light onto the surface of the object to be measured by the one or more light projectors; a first acquisition module for acquiring an image of the surface using the first multi-view camera; and a first generation module for generating a three-dimensional shape of the surface using the image of the surface.

[0076] Optionally, the scanner further includes one or more light projectors and a second multi-view camera located on the first surface of the housing. The device further includes: a second projection module for projecting structured light onto the surface of the object to be measured by the one or more light projectors; a second acquisition module for acquiring an image of the surface using the second multi-view camera; and a second generation module for generating a three-dimensional shape of the surface using the image of the surface.

[0077] The scanner calibration device provided in this disclosure can execute the scanner calibration method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects of executing the method.

[0078] This disclosure also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the scanner calibration method provided in any embodiment of this disclosure.

[0079] Figure 5 This is a schematic diagram of an electronic device provided in an embodiment of the present disclosure. See below for details. Figure 5 The diagram illustrates a structural schematic suitable for implementing the electronic device 500 in the embodiments of this disclosure. The electronic device 500 in the embodiments of this disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0080] like Figure 5As shown, the electronic device 500 may include a processing unit 501 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the electronic device 500. The processing unit 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0081] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 508 including, for example, magnetic tapes, hard disks, etc.; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0082] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by the processing device 501, it performs the functions defined in the scanner calibration method of embodiments of this disclosure.

[0083] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0084] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0085] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0086] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: use a first multi-view camera to capture at least a portion of a first mark of an extended frame, thereby obtaining a multi-frame image to be processed that includes at least a portion of the first mark; and process the multi-frame image to obtain the relative positional relationship between the scanner and the tracking frame.

[0087] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0088] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0089] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0090] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0091] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0092] According to one or more embodiments of this disclosure, this disclosure provides an electronic device, including: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement any of the scanner calibration methods provided in this disclosure.

[0093] According to one or more embodiments of the present disclosure, the present disclosure provides a computer-readable storage medium storing a computer program for performing any of the scanner calibration methods provided in the present disclosure.

[0094] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0095] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0096] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A scanner calibration method, the scanner being detachably connected to a tracking frame, the scanner including a housing and a first multi-view camera located on a first surface of the housing, the tracking frame including a main frame and an extension frame, the extension frame including a plurality of first marks, wherein when the scanner is connected to the main frame, the first multi-view camera faces outward from the main frame and the field of view of the first multi-view camera covers at least a portion of the first marks, the method comprising: The first multi-view camera is used to capture at least a portion of the first marker of the extended frame to obtain a multi-frame image to be processed, including the at least a portion of the first marker; The relative positional relationship between the scanner and the tracking frame is obtained by processing the multiple frames of images to be processed.

2. The method according to claim 1, wherein, The plurality of first markers have real coordinate values. Based on the multiple frames of images to be processed, the relative positional relationship between the scanner and the tracking frame is obtained, including: Based on the multiple frames of images to be processed, multiple first three-dimensional coordinate points reconstructed from at least some of the first markers are obtained; The first rigid body transformation relationship between the scanner and the tracking frame is determined by calculation based on the real coordinate values ​​of the plurality of first three-dimensional coordinate points and at least some of the first marks, as the relative positional relationship.

3. The method according to claim 2, wherein, Processing is performed on the multiple frames of images to be processed to obtain multiple first three-dimensional coordinate points reconstructed from at least a portion of the first markers, including: Obtain the two-dimensional coordinates of at least a portion of the first marker in each frame of the image to be processed; Based on the relative positions between the first multi-view cameras and the two-dimensional coordinate points, the three-dimensional coordinate points of at least some of the first markers in each frame of the image to be processed are reconstructed. Based on the common first marker in each frame of the image to be processed, the multiple first three-dimensional coordinate points in the same coordinate system are obtained by stitching together.

4. The method according to claim 1, wherein, The main frame includes a main structure and multiple targets, which are respectively located on the outside of multiple joints of the main structure. Each target includes multiple second markers, which have real coordinate values. The method further includes: The tracking device is used to track at least a portion of the second markers on the plurality of targets to determine the position of the tracking frame during movement; The position of the scanner during the movement is determined based on the position of the tracking frame and the relative positional relationship.

5. The method according to claim 1, further comprising: The scanner's scanning space is monitored, and in response to a mismatch between the scanning space and the tracking frame, a prompt to remove the tracking frame is generated.

6. The method of claim 5, further comprising, after removing the tracking frame: In response to the scan space being adapted to the tracking frame, a prompt to connect to the tracking frame is generated.

7. The method according to claim 6, further comprising: After each connection to the tracking frame, the relative positional relationship between the scanner and the tracking frame is reacquired.

8. The method according to any one of claims 1-7, wherein, The first multi-view camera is a stereo camera.

9. The method according to any one of claims 1-7, wherein, The scanner further includes one or more light projectors located on the first surface of the housing, and the method further includes: The one or more light projectors project structured light onto the surface of the object under test; The first multi-view camera is used to acquire an image of the surface; The three-dimensional shape of the surface is generated using an image of the surface.

10. The method according to any one of claims 1-7, wherein, The scanner further includes one or more light projectors and a second multi-view camera located on the first surface of the housing, and the method further includes: The one or more light projectors project structured light onto the surface of the object under test; The second multi-view camera is used to acquire an image of the surface; The three-dimensional shape of the surface is generated using an image of the surface.

11. An electronic device, comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the scanner calibration method according to any one of claims 1-10.

12. A scanner, comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the scanner calibration method according to any one of claims 1-10.

13. A computer-readable storage medium, characterized in that, The storage medium stores a computer program for executing the scanner calibration method according to any one of claims 1-10.

Images