Calibration method, calibration device, computer program product, equipment and storage medium

Abstract

The embodiments of the present disclosure provide a calibration method, calibration device, computer program product, device and storage medium for a three-dimensional scanning device. A large calibration plate can be obtained by combining a marker and at least two calibration plates, wherein the marker includes a plurality of marker points with known relative position relationships, and the calibration plate includes a plurality of marker points with relative position relationships, and then the three-dimensional scanning device is calibrated using the large calibration plate. By using this "marker + calibration plate" combination to obtain a large calibration plate, only a small number of marker points with known positions need to be set when making the marker, and the positions of the marker points in the calibration plate do not need to be determined in advance. Therefore, while ensuring the calibration accuracy, the production and use of the combined large calibration plate can be made more convenient.

Description

Technical Field

[0001] The present disclosure relates to the field of three-dimensional scanning technology, and in particular to a calibration method, calibration device, computer program product, device, and storage medium for a three-dimensional scanning device. Background Art

[0002] Before use, 3D scanning devices typically use a calibration plate to calibrate the intrinsic and extrinsic parameters of the camera in the 3D scanning device. When using a calibration plate to calibrate a 3D scanning device, it is typically required that the plate cover the device's scanning angle of view. Therefore, for some 3D scanning devices with a large scanning angle of view, a large calibration plate is typically used for calibration. However, large calibration plates made of lightweight materials are prone to deformation, which cannot guarantee the accuracy of the calibrated intrinsic and extrinsic parameters. Therefore, large calibration plates are typically made of non-deformable materials such as marble. These plates are heavy and inconvenient to use, and large calibration plates are also inconvenient to transport. Of course, a small calibration plate can also be used to calibrate 3D scanning devices with a large scanning angle of view. However, when using a small calibration plate, to ensure that the plate covers the device's scanning angle of view, the 3D scanning device must collect calibration images from more locations. Typically, the number of calibration images collected is about four times the number of calibration images collected when using a large calibration plate. Obviously, the significantly increased number of calibration images collected significantly increases the computational workload in the subsequent image processing stage, reducing calibration efficiency.

[0003] Therefore, it is necessary to provide a calibration solution that is convenient for transportation and use of the calibration plate, reduces the number of calibration images collected during the calibration process, and reduces the amount of calculation. Summary of the Invention

[0004] The present disclosure provides a calibration method, a calibration device, a computer program product, a device, and a storage medium for a three-dimensional scanning device.

[0005] According to a first aspect of an embodiment of the present disclosure, a calibration method for a three-dimensional scanning device is provided, the method comprising:

[0006] Using at least two cameras to be calibrated in the three-dimensional scanning device to respectively capture images of the calibration device to obtain multiple frames of calibration images; wherein the calibration device includes a marker and at least two calibration plates placed near the marker, the marker includes a plurality of first marker points whose relative positional relationship is known, and each of the calibration plates includes a plurality of second marker points whose relative positional relationship is unknown;

[0007] Performing three-dimensional reconstruction on the calibration device based on initial intrinsic parameters of each of the at least two cameras, initial extrinsic parameters between the at least two cameras, and the multiple frames of calibration images to obtain reconstructed three-dimensional coordinates of each of the first marker point and the second marker point in the multiple frames of calibration images;

[0008] Based on the reconstructed three-dimensional coordinates, the reprojection errors of the first marker point and the second marker point in the multiple frames of calibration images are determined, and the initial extrinsic parameters are optimized based on the reprojection errors and the relative position relationship between the first marker points to obtain the calibration extrinsic parameters between the at least two cameras.

[0009] According to a second aspect of an embodiment of the present disclosure, a computer program product is provided, wherein the computer program product includes a computer program, and when the computer program is executed, the method mentioned in the first aspect is implemented.

[0010] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising a processor, a memory, and computer instructions stored in the memory for execution by the processor. When the processor executes the computer instructions, the method mentioned in the first aspect above can be implemented.

[0011] According to a fourth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, on which computer instructions are stored. When the computer instructions are executed, the method mentioned in the first aspect is implemented.

[0012] According to a fifth aspect of an embodiment of the present disclosure, a calibration device is provided, which includes a marker and at least two calibration plates, wherein the marker includes a plurality of first marker points whose relative position relationships are known, and each of the calibration plates includes a plurality of second marker points whose relative position relationships are unknown; wherein, when the calibration device is used to calibrate a device, the at least two calibration plates are placed near the marker to combine to obtain a large-sized combined calibration plate.

[0013] In the embodiment of the present disclosure, a large calibration plate can be obtained by combining a marker and at least two calibration plates, and the large calibration plate obtained by the combination can be used to calibrate the three-dimensional scanning device. Among them, some marker points with known positions can be pre-set in the marker, and some marker points can be set arbitrarily in the calibration plate, and the positions of these marker points do not need to be known. Since the positions of the marker points in the calibration plate do not need to be determined in advance, it is convenient to make, and thus a large number of feature points can be provided for stereo matching in the subsequent calibration process by using a combination of multiple calibration plates to improve the matching accuracy. A small number of marker points with known positions can be set in the marker for further calibration of external parameters to improve the calibration accuracy. By using this "marker + calibration plate" combination to obtain a large calibration plate, only a small number of marker points with known positions need to be set when making the marker, and the positions of the marker points in the calibration plate do not need to be determined in advance. Therefore, while ensuring the calibration accuracy, the production of the combined large calibration plate is made simpler and more convenient.

[0014] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings herein are incorporated into and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and, together with the specification, are used to explain the technical solutions of the present disclosure.

[0016] Figure 1 This is a flow chart of a three-dimensional scanning device calibration method according to an embodiment of the present disclosure.

[0017] Figure 2 It is a schematic diagram of a calibration device according to an embodiment of the present disclosure.

[0018] Figure 3 Schematic diagram of a marker according to an embodiment of the present disclosure.

[0019] Figure 4 Schematic diagram of another calibration device according to an embodiment of the present disclosure.

[0020] Figure 5 This is a schematic diagram of an embodiment of the present disclosure for collecting calibration images using a three-dimensional scanning device.

[0021] Figure 6 It is a schematic diagram of the logical structure of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0022] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

[0023] The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. The singular forms "a", "the" and "the" used in this disclosure and the appended claims are also intended to include plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items. In addition, the term "at least one" herein means any combination of at least two of any one or more of a plurality of.

[0024] It should be understood that although the terms first, second, third, etc. may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining."

[0025] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present disclosure and to make the above-mentioned purposes, features and advantages of the embodiments of the present disclosure more obvious and easy to understand, the technical solutions in the embodiments of the present disclosure are further described in detail below with reference to the accompanying drawings.

[0026] Some 3D scanning devices include a binocular vision system, which uses two cameras in the binocular vision system to obtain two images of the same scene from different perspectives. The depth information of the scene can be restored through the parallax of the two images and the internal and external parameters of the two cameras to achieve 3D reconstruction. Although the internal and external parameters of the two cameras are pre-calibrated when the 3D scanning device leaves the factory. However, during the use of the 3D scanning device, various factors such as temperature and external force may cause the structural parts that fix the camera to deform or shift in position, causing the external parameters of the two cameras to change. In order to obtain more accurate 3D reconstruction results, before using the 3D scanning device to perform 3D reconstruction of the target to be reconstructed, it is usually necessary to recalibrate the external parameters of the two cameras in the 3D scanning device, and use the calibrated external parameters to obtain the 3D reconstruction results.

[0027] Typically, the extrinsic parameters of the two cameras in a 3D scanning device can be calibrated using a pre-set calibration plate. This plate typically consists of multiple feature points or targets (such as black and white grids, spherical or circular markers, or a checkerboard pattern) with known positions and sizes. The size and shape of the calibration plate must be accurate and stable. During the calibration process, the calibration plate can be scanned multiple times from different angles and positions using a 3D scanner to obtain multiple frames of calibration images. Since the 3D coordinates of the feature points on the calibration plate (such as the corners of the black and white grid or the center of a circular marker) are known, the extrinsic parameters of the two cameras can be determined based on the 2D coordinates of these feature points in the calibration image and their 3D coordinates.

[0028] When using a calibration plate to calibrate a 3D scanning device, it's typically required to cover the device's field of view (FOV) (i.e., the FOV of both cameras). For example, to obtain accurate calibration results, the calibration plate should ideally occupy 70% to 80% of the calibration image captured by the 3D scanning device's cameras. Therefore, for 3D scanning devices with large field of view, large calibration plates are often used. However, large calibration plates made of lightweight materials are prone to deformation, making it difficult to guarantee the accuracy of the calibrated internal and external parameters. Consequently, large calibration plates are often made of non-deformable materials like marble. These heavy plates are inconvenient to use and transport. Of course, a small-sized calibration plate can also be used to calibrate a 3D scanning device with a large scanning angle. However, when using a small-sized calibration plate, in order to ensure that the calibration plate covers the scanning angle of the 3D scanning device, the 3D scanning device needs to be moved to more positions to capture images of the small-sized calibration plate. Usually, the number of calibration images captured is about 4 times the number of calibration images captured when using a large calibration plate. Obviously, when the number of calibration images captured is greatly increased, the computational complexity of the subsequent image processing stage will be greatly increased, thereby reducing the calibration efficiency.

[0029] Therefore, it is necessary to provide a calibration solution that is convenient for transportation and use of the calibration plate, reduces the number of images collected during the calibration process, and reduces the amount of calculation.

[0030] Based on this, the present invention proposes that a large-scale calibration plate can be obtained by combining multiple small-scale calibration plates, thereby facilitating the transportation of the calibration plates and reducing the number of calibration images required. Furthermore, considering that for each small-scale calibration plate, if feature points need to be set in advance in each small-scale calibration plate and the positions of these feature points are determined, the production of the calibration plate will be very cumbersome due to the large number of feature points. Therefore, in order to further simplify the production of the calibration plate, the present invention proposes that a large-scale calibration plate can be obtained by combining a marker and at least two calibration plates, and using the large-scale calibration plate obtained by this combination to calibrate the three-dimensional scanning device. Among them, some marker points with known positions can be pre-set in the marker, while some marker points can be set arbitrarily in the calibration plate, and the positions of these marker points do not need to be known. Since the positions of the marker points in the calibration plate do not need to be determined in advance, it is convenient to produce. Therefore, a large-scale calibration plate can be obtained by combining multiple calibration plates, so that more feature points are distributed in the images collected by the camera to be calibrated, providing a large number of feature points for stereo matching in the subsequent calibration process, thereby improving matching accuracy and obtaining more accurate internal and external parameters. A small number of marker points with known positions can be set in the marker to further calibrate the external parameters and improve the calibration accuracy.

[0031] This "marker + calibration plate" combination creates a large calibration plate. Only a small number of known marker points need to be set when creating the marker, while the positions of the marker points on the calibration plate do not need to be predetermined. This makes the production of the combined large calibration plate simpler and more convenient while ensuring calibration accuracy. Furthermore, when providing users with a calibration device for calibrating 3D scanning equipment, only the "marker" can be provided, while the calibration plate can be completely made by the user, making transportation of the calibration plate more convenient.

[0032] Based on the above-described inventive concepts, embodiments of the present application provide a calibration method for a 3D scanning device. The calibration method can be performed by the 3D scanning device. For example, the 3D scanning device can determine the extrinsic parameters of two cameras based on captured calibration images. Alternatively, the method can be performed by other electronic devices (e.g., mobile phones, tablets, computers, etc.) that are communicatively connected to the 3D scanning device. For example, the 3D scanning device can send captured calibration images to these devices, which can then determine the extrinsic parameters of the two cameras based on the calibration images.

[0033] like Figure 1 As shown, the method may include the following steps:

[0034] S102, using at least two cameras to be calibrated in the three-dimensional scanning device to respectively capture images of the calibration device to obtain multiple frames of calibration images; wherein the calibration device includes a marker and at least two calibration plates placed near the marker, the marker includes a plurality of first marker points whose relative positional relationship is known, and each of the calibration plates includes a plurality of second marker points whose relative positional relationship is unknown;

[0035] Before calibrating the 3D scanning device, the user can first use the combination of "marker + calibration plate" to obtain a calibration device for calibrating the 3D scanning device, such as Figure 2 FIG2 is a schematic diagram of a calibration device in one embodiment of the present application, wherein the shape of the marker, the number of calibration plates, and the arrangement of the calibration plates are not limited to those shown in the figure. The marker may include multiple pre-set marker points, hereinafter referred to as first marker points, wherein the relative positional relationships between the multiple first marker points are known. The shape of the marker can be a square, a strip, a circle, or other shapes, which are not limited in the present embodiment. The number, size, and arrangement of the first marker points can also be set based on actual needs. For example, they can be evenly distributed within the marker or unevenly distributed. The sizes of the multiple first marker points can be consistent or inconsistent, depending on actual needs. Since the marker needs to serve as a "ruler" to calibrate the external parameters of the two cameras during calibration, the relative positional relationships between the multiple first marker points in the marker need to be predetermined. For example, in actual use, the true three-dimensional coordinates of each first marker point can be stored. Of course, if the first marker points are arranged in an array, only the distances between the first marker points can be stored, and the true three-dimensional coordinates of each first marker point can be obtained based on these distances.

[0036] Multiple marking points can also be pre-set in the calibration plate, hereinafter referred to as second marking points. The relative positional relationship between the second marking points does not need to be determined in advance, and therefore, the setting of the second marking points can be relatively arbitrary. The distribution mode of the second marking points in the calibration plate, the size and number of the second marking points can also be set based on actual needs. For example, the distribution mode can be random distribution or array distribution, and the size of each second marking point can be consistent or inconsistent, which is not limited in this application. In addition, the distribution mode, size and number of the second marking points in different calibration plates can be the same or different.

[0037] The positions of the second markers in the calibration plate do not need to be predetermined, making their production relatively simple. Therefore, their number can be larger, providing more feature points for stereo matching in the subsequent calibration process, thereby improving matching accuracy. The positions of the first markers in the marker object do need to be predetermined, making the production process more complex. Therefore, the number of first markers can be smaller, and further calibration of the external parameters can be performed using a small number of markers with known positions to improve calibration accuracy.

[0038] The calibration plates can be placed on both sides of the marker, that is, the marker should be placed in the center of the calibration device as much as possible so that each frame of the calibration image collected includes the first marker point.

[0039] After obtaining a calibration device using the "marker + calibration plate" combination, the calibration device can be used to calibrate two or more cameras to be calibrated in the 3D scanning device to determine the extrinsic parameters between any two cameras. In particular, for scenes with three or more cameras to be calibrated, calibration can be performed on each of them first to determine the relative extrinsic parameters between any two cameras. Of course, it is also possible to calibrate the three or more cameras simultaneously, and the specific settings can be flexibly set based on actual needs.

[0040] During the calibration process, at least two cameras to be calibrated can be used to capture images of the calibration device, generating multiple frames of calibration images. Furthermore, a 3D scanning device can be used to capture images of the calibration device at different positions and angles, generating calibration images at different positions and angles, so that these calibration images cover as many areas of the calibration device as possible.

[0041] S104, based on the initial intrinsic parameters of each of the at least two cameras, the initial extrinsic parameters between the at least two cameras, and the multiple frames of calibration images, the calibration device is three-dimensionally reconstructed to obtain the reconstructed three-dimensional coordinates of the first marker point and the second marker point in the multiple frames of calibration images; in step S104, since the intrinsic parameters and extrinsic parameters of any two cameras in the three-dimensional scanning device are calibrated before leaving the factory, or are pre-calibrated before each use. Therefore, after acquiring multiple frames of calibration images, the calibration device can be three-dimensionally reconstructed based on the initial intrinsic parameters of each of the at least two cameras, the initial extrinsic parameters between the at least two cameras, and the multiple frames of calibration images to obtain the reconstructed three-dimensional coordinates of the first marker point and the second marker point in the multiple frames of calibration images. Among them, the initial intrinsic parameters and initial extrinsic parameters can be the default internal parameters and extrinsic parameters at the time of leaving the factory, or the internal parameters and extrinsic parameters determined by the last calibration, and the embodiments of the present application do not limit this.

[0042] Because at least two cameras simultaneously capture images of the calibration device, stereo matching can be performed on two frames of calibration images captured simultaneously by the at least two cameras to determine the depth information of the first and second marker points in the calibration images. Furthermore, for calibration images captured by the same camera at different times, inter-frame matching can also be performed based on the overlapping regions in these calibration images to obtain the reconstructed 3D coordinates of the first and second marker points in these multiple frames of calibration images.

[0043] S106. Determine the reprojection errors of the first marker point and the second marker point in the multiple frames of calibration images based on the reconstructed three-dimensional coordinates, optimize the initial extrinsic parameters based on the reprojection errors and the relative positional relationship between the first marker points, and obtain the calibration extrinsic parameters between the at least two cameras.

[0044] In step S106, after obtaining the reconstructed three-dimensional coordinates of the first marker point and the second marker point in the multiple frames of calibration images, the reprojection errors of the first marker point and the second marker point can be determined based on the reconstructed three-dimensional coordinates. Then, the initial extrinsic parameters are optimized using the reprojection errors and the relative positional relationship between the first marker points as constraints to obtain the calibration extrinsic parameters between at least two cameras.

[0045] For example, in some embodiments, when optimizing initial extrinsic parameters based on reprojection errors and the relative positional relationship between first marker points to obtain calibration extrinsic parameters between at least two cameras, the initial extrinsic parameters can be optimized with the goal of minimizing the reprojection error between the first marker points and the second marker points to obtain optimized extrinsic parameters. For example, based on the initial extrinsic parameters, the extrinsic parameters can be continuously adjusted (increasing or decreasing the initial extrinsic parameters by a certain gradient), and then the reconstructed three-dimensional coordinates can be reprojected onto the calibration image based on the adjusted extrinsic parameters. The reprojection errors of each of the first marker points and the second marker points can be determined, and the extrinsic parameters with the minimum reprojection error can be used as the optimized extrinsic parameters. Of course, given the errors in stereo matching and inter-frame matching during the three-dimensional reconstruction process, the optimized extrinsic parameters obtained in this way are not yet accurate enough. Therefore, the optimized extrinsic parameters can be further calibrated and constrained based on the relative positional relationship between any two first marker points, so that the final calibration extrinsic parameters have higher accuracy. Specifically, the three-dimensional coordinates of the first marker points in the multiple calibration image frames can be determined based on the optimized extrinsic parameters, hereinafter referred to as the optimized three-dimensional coordinates. Since the relative positional relationships between the first marker points are known, the true three-dimensional coordinates of each first marker point can be obtained. The optimized extrinsic parameters can then be further optimized based on the difference between the optimized three-dimensional coordinates and the true three-dimensional coordinates of the first marker points in the multiple calibration image frames to obtain the calibration extrinsic parameters between the two cameras. For example, the optimization objective can be to minimize the sum of the deviations between the optimized three-dimensional coordinates and the true three-dimensional coordinates of the multiple first marker points, and the calibration parameters can be obtained by optimizing the optimized parameters.

[0046] First, the initial extrinsic parameters are optimized using the reprojection errors of a large number of feature points (i.e., the first landmark point and the second landmark point), so that relatively accurate optimized extrinsic parameters can be obtained. Then, the optimized extrinsic parameters can be further calibrated based on the true three-dimensional coordinates of the first landmark point, so that highly accurate calibration extrinsic parameters can be obtained.

[0047] Of course, in some embodiments, the initial extrinsic parameter can also be calibrated based on the relative position relationship between the first marker points. For example, the real three-dimensional coordinates of these first marker points can be determined based on the above relative position relationship. After obtaining the reconstructed three-dimensional coordinates of the first marker point in the multi-frame calibration image, the initial extrinsic parameter is calibrated based on the difference between the reconstructed three-dimensional coordinates of the first marker point and the real three-dimensional coordinates to obtain the optimized extrinsic parameter. Then, the optimized extrinsic parameter and the reconstructed three-dimensional coordinates of the first marker point and the second marker point are used to determine the reprojection error of the first marker point and the second marker point in the multi-frame calibration image. Minimizing the reprojection error is used as the optimization goal, and the optimized extrinsic parameter is further optimized to obtain the calibration extrinsic parameter.

[0048] In some embodiments, an optimization function can also be constructed based on the reprojection errors of the first marker point and the second marker point, and the difference between the optimized three-dimensional coordinates and the true three-dimensional coordinates of the first marker point. For example, the sum of the two is used as the optimization target, and then the initial extrinsic parameter is continuously adjusted to minimize the sum of the two, and the extrinsic parameter at this time is used as the calibration extrinsic parameter.

[0049] Among them, there are many specific ways to use the reprojection error and the relative position relationship between any two first landmark points as the initial external parameters for constrained optimization, which can be set based on actual needs and are not limited in the embodiments of this application.

[0050] In some instances, each first marker point in the marker corresponds to an identifier, which can be used to uniquely identify the first marker point. For example, the identifier can be the marker point number. The identifier of each first marker point is stored in association with the real three-dimensional coordinates of the first marker point. The real three-dimensional coordinates of each first marker point are determined based on the relative positional relationship between any two first marker points. In addition, in order to distinguish the various first marker points in the calibration image and determine the real three-dimensional coordinates corresponding to each first marker point in the calibration image, when setting the first marker points in the marker, the multiple first marker points can be distinguished by visual information.

[0051] In some embodiments, these visual information may include one or more of the following: the size of the first marker point, the shape of the first marker point, the color of the first marker point, the arrangement of the first marker point, and the marking information located around the first marker point. For example, the size, shape, and color of different first marker points may be different, and this information can be stored in association with the identifier of the first marker point, so that after obtaining the calibration image, the size, shape, color, etc. of the first marker point in the calibration image can be identified to determine the identifier of the first marker point, and then determine the true three-dimensional coordinates of the first marker point. Of course, some marks can also be made around the first marker point, such as marking the number of the first marker point, so that after obtaining the calibration image, the number of the first marker point in the calibration image can be identified to determine the true three-dimensional coordinates of the first marker point.

[0052] In some embodiments, to facilitate the configuration of first marker points in a marker, the multiple first marker points in the marker can be divided into multiple groups, with the first marker points in each group arranged differently. That is, by configuring the arrangement of the first marker points in each group to be different, after acquiring a calibration image, the current arrangement of the first marker points can be identified from the calibration image to determine which group it belongs to, and the first marker points within each group can be further distinguished based on their position, size, color, etc.

[0053] In some embodiments, the marker can be a long strip, and the first marker points of the multiple groups can be evenly spaced along the long side of the marker. That is, the groups are evenly spaced within the marker to ensure that the first marker points are included in the calibration images captured when the 3D scanning device is moved to different locations.

[0054] For example, Figure 3 Figure 2 shows a schematic diagram of a marker in one embodiment of the present application. The marker is a long strip, and three groups of first marker points are evenly spaced. Each group includes a large first marker point and several small first marker points. The arrangement of the large first marker points and the several small first marker points in different groups varies. After acquiring a calibration image, the group to which the first marker point belongs can be determined based on the arrangement of the first marker points. The first marker points in the group can then be further distinguished and numbered based on their size and relative position.

[0055] In some embodiments, the marker is an elongated strip, and the at least two calibration plates can be symmetrically arranged along the long side of the marker. Specifically, the marker is arranged in an elongated strip, and the calibration plates are symmetrically arranged along the long side of the elongated strip. This makes the resulting calibration device more compact and occupies less space. Furthermore, the marker is positioned at the center of the entire calibration device, so that subsequently acquired calibration images can include the first marker point, allowing the relative position of the first marker point to be used to constrain and calibrate the 3D points reconstructed in the calibration image.

[0056] Considering that if the number of calibration plates in the calibration device is too small, the scanning angle of the three-dimensional scanning device cannot be covered, and more calibration images need to be collected subsequently, which increases the amount of calculation. If the number of calibration plates is too large and exceeds the actual required number, it will increase the user operations during the calibration process, making the calibration process more cumbersome. Therefore, in some embodiments, in order to make the number of calibration plates in the calibration device reasonable, the number of calibration plates can be determined based on the scanning angle of the three-dimensional scanning device. The larger the scanning angle, the more calibration plates, that is, the number of calibration plates is positively correlated with the scanning angle of the three-dimensional scanning device. Of course, in some embodiments, under the premise that the size of the calibration plate is fixed, the mapping relationship between the scanning angle of the three-dimensional scanning device and the number of calibration plates can be set in advance, or the mapping relationship between the type of three-dimensional scanning device and the number of calibration plates can be set, so that the user can directly place the appropriate number of calibration plates based on the mapping relationship during calibration.

[0057] In some embodiments, as Figure 4As shown, considering the scanning angle of existing three-dimensional scanning equipment, the number of the calibration plates can be 6, and 3 calibration plates can be placed on each side of the marker, and the 3 calibration plates can be placed symmetrically on both sides of the marker.

[0058] In some embodiments, in order to obtain accurate calibration external parameters based on the collected calibration images, the multiple frames of calibration images can be images collected when the 3D scanning device is located at different positions and at different angles to the calibration device. For example, Figure 5 As shown, the 3D scanning device can be aligned with the center of the marker, moved up and down to several positions along a direction perpendicular to the plane of the calibration plate, and an image is captured at each position. Then, the 3D scanning device can be aligned with the calibration plates placed on both sides of the marker, moved to several positions along a direction parallel to the plane of the calibration plates, and at each position, the 3D scanning device can be controlled to form a certain angle with the plane of the calibration plates in the front, back, left, and right directions to capture an image. Each frame of the calibration image captured includes the first marker point of the marker.

[0059] For example, Figure 4 As shown in the figure, assuming that the marker is a long strip marker with three calibration plates placed on each side of the marker, the acquisition process for the calibration image is as follows:

[0060] (1) Align the 3D scanning device with the marker and calibrate the device plane vertically. Move the 3D scanning device up and down to 5 different positions and collect a calibration image at each position.

[0061] (2) Aim the 3D scanning device at the three calibration plates on one side of the marker, maintaining a 20-degree angle with the plane of the calibration plates. Collect 5 positions in each tilt direction, for a total of 20 sets of images.

[0062] (3) Aim the 3D scanning device at the three calibration plates on the other side of the marker, maintaining a 20-degree angle with the plane of the calibration plates. Collect 5 positions in each tilt direction, for a total of 20 sets of images.

[0063] In some embodiments, the at least two cameras to be calibrated may be two black-and-white cameras in a 3D scanning device, wherein the 3D scanning device further includes a texture camera for collecting texture information of the object to be reconstructed. After obtaining the calibration extrinsic parameters between the two black-and-white cameras, since the calibration extrinsic parameters are relatively accurate, the three-dimensional coordinates of the first and second marker points in the calibration device can be further determined based on the calibration extrinsic parameters. This is equivalent to having a large calibration plate with known coordinates for each marker point. This large calibration plate with known coordinates for each marker point can then be used to calibrate the extrinsic parameters between the black-and-white camera and the texture camera. Specifically, for each of the two black-and-white cameras, the calibration device can be used to capture images using the black-and-white camera and the texture camera, respectively, to obtain multiple frames of calibration images. The three-dimensional coordinates of each first and second marker point in the calibration images are known. Calibration extrinsic parameters for the black-and-white camera and the texture camera can then be determined based on the two-dimensional coordinates of the first and second marker points in the calibration images and the three-dimensional coordinates.

[0064] In addition, an embodiment of the present application further provides a calibration device, comprising a marker and at least two calibration plates, wherein the marker includes a plurality of first marker points whose relative positional relationships are known, and each calibration plate includes a plurality of second marker points whose relative positional relationships are unknown. When the calibration device is used to calibrate a device, the at least two calibration plates are placed near the marker to form a large-scale combined calibration plate. By combining the marker and multiple calibration plates to form a large calibration plate, the size and weight of each calibration plate can be reduced, facilitating transportation and use of the calibration device.

[0065] In some embodiments, multiple first marker points in the marker can be distinguished by visual information, each first marker point corresponds to an identifier, and the identifier of each first marker point is stored in association with the real three-dimensional coordinates of the first marker point, wherein the real three-dimensional coordinates of each first marker point are determined based on the relative position relationship between the first marker points.

[0066] In some embodiments, the visual information includes one or more of the following: the size of the first marker point, the shape of the first marker point, the color of the first marker point, the arrangement of the first marker point, and marking information located around the first marker point.

[0067] In some embodiments, the plurality of first marker points in the marker are divided into a plurality of groups, and the first marker points in the plurality of groups are arranged in different ways.

[0068] In some embodiments, the marker is a long strip-shaped marker. When the calibration device is used to calibrate the device, the at least two calibration plates are symmetrically placed along the long side of the marker.

[0069] In some embodiments, the number of the calibration plates is 6, with 3 calibration plates placed on each side of the marker.

[0070] The specific structure of the calibration device can be referred to the description in the above embodiment and will not be repeated here.

[0071] It is not difficult to understand that the solutions described in the above embodiments can be freely combined to obtain new solutions when there is no conflict. Due to space reasons, they are not listed one by one in the embodiments of this disclosure.

[0072] Accordingly, an embodiment of the present disclosure further provides a computer program product, which includes a computer program. When the computer program is executed, the method described in any one of the above embodiments is implemented.

[0073] Furthermore, the present disclosure also provides an electronic device, such as Figure 6 As shown, the device includes a processor 61, a memory 62, and computer instructions stored in the memory 62 for execution by the processor 61. When the processor 61 executes the computer instructions, the method described in any one of the above embodiments is implemented.

[0074] An embodiment of the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, which implements the method described in any of the aforementioned embodiments when the program is executed by a processor.

[0075] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can be implemented using any method or technology to store information. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change RAM (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0076] Through the above description of the embodiments, it can be seen that those skilled in the art can clearly understand that the embodiments of the present disclosure can be implemented by means of software plus the necessary general hardware platform. Based on this understanding, the technical solutions of the embodiments of the present disclosure, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments of the present disclosure.

[0077] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email transceiver, game console, tablet computer, wearable device, or any combination of these devices.

[0078] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described above is merely illustrative, wherein the modules described as separate components may or may not be physically separated, and the functions of each module can be implemented in the same one or more software and / or hardware when implementing the embodiment of the present disclosure. It is also possible to select some or all of the modules according to actual needs to achieve the purpose of the embodiment. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0079] The above is only a specific implementation of the embodiment of the present disclosure. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the embodiment of the present disclosure. These improvements and modifications should also be regarded as the scope of protection of the embodiment of the present disclosure.

Claims

1. A calibration method for a three-dimensional scanning device, characterized in that: The method comprises: Utilize at least two cameras to be calibrated in the three-dimensional scanning device to respectively acquire images of the calibration device to obtain multiple frames of calibration images; wherein, the calibration device includes a marker and at least two calibration plates placed near the marker, the marker includes a plurality of first marker points, the real three-dimensional coordinates of each of the plurality of first marker points have been predetermined and stored in the device that executes the calibration method, each first marker point corresponds to an identifier that is used to uniquely identify the first marker point, and the identifier is stored in association with the real three-dimensional coordinates of the first marker point, each of the calibration plates includes a plurality of second marker points whose relative positional relationships are unknown, wherein each frame of the calibration image includes one or more of the first marker points; Performing three-dimensional reconstruction on the calibration device based on initial intrinsic parameters of each of the at least two cameras, initial extrinsic parameters between the at least two cameras, and the multiple frames of calibration images to obtain reconstructed three-dimensional coordinates of each of the first marker point and the second marker point in the multiple frames of calibration images; Based on the reconstructed three-dimensional coordinates, the reprojection errors of the first marker point and the second marker point in the multiple frames of calibration images are determined, and the initial extrinsic parameters are optimized based on the reprojection errors and the true three-dimensional coordinates of the multiple first marker points to obtain the calibration extrinsic parameters between the at least two cameras.

2. The method according to claim 1, characterized in that Optimizing the initial extrinsic parameters based on the reprojection error and the relative positional relationship between the first marker points to obtain calibration extrinsic parameters between the at least two cameras includes: The initial extrinsic parameters are optimized with minimizing the reprojection error as an optimization goal to obtain optimized extrinsic parameters; based on the optimized extrinsic parameters, optimized three-dimensional coordinates of a first marker point in the multiple calibration image frames are determined; based on the difference between the optimized three-dimensional coordinates of the first marker point in the multiple calibration image frames and the true three-dimensional coordinates of the first marker point, the optimized extrinsic parameters are further optimized to obtain calibration extrinsic parameters between the at least two cameras; or Optimizing the initial extrinsic parameters based on a difference between the reconstructed three-dimensional coordinates of the first marker point in the multiple frames of calibration images and the true three-dimensional coordinates of the first marker point to obtain optimized extrinsic parameters; determining a reprojection error of each of the first marker point and the second marker point in the multiple frames of calibration images based on the optimized extrinsic parameters and the reconstructed three-dimensional coordinates of each of the first marker point and the second marker point in the multiple frames of calibration images; and further optimizing the optimized extrinsic parameters with minimizing the reprojection error as an optimization goal to obtain calibration extrinsic parameters between the at least two cameras; The real three-dimensional coordinates are determined based on the relative position relationship between the first marker points.

3. The method according to claim 1, characterized in that The plurality of first marker points in the marker can be distinguished by visual information.

4. The method according to claim 3, characterized in that The visual information includes one or more of the following: the size of the first marker point, the shape of the first marker point, the color of the first marker point, the arrangement of the first marker point, and marking information located around the first marker point; and / or The plurality of first marking points in the marker are divided into a plurality of groups, and the arrangement manners of the first marking points in the plurality of groups are different.

5. The method according to claim 4, characterized in that The marker is a long strip-shaped marker, and the multiple groups are distributed at equal intervals along the long side of the marker.

6. The method according to claim 1, wherein The marker is a long strip marker, and the at least two calibration plates are symmetrically placed along the long side of the marker; and / or The number of the calibration plates is positively correlated with the scanning angle of the three-dimensional scanning device.

7. The method according to claim 6, characterized in that The number of the calibration plates is 6, and 3 calibration plates are placed on each side of the marker.

8. The method according to claim 1, characterized in that The multi-frame calibration image includes: Aligning the three-dimensional scanning device with the center of the marker, moving it to several positions along a direction perpendicular to the plane of the calibration plate, and capturing an image at each position; and The three-dimensional scanning device is aimed at the calibration plates placed on both sides of the marker, and moved to several positions along the direction parallel to the plane where the calibration plates are located. At each position, the three-dimensional scanning device is controlled to form a certain angle with the plane where the calibration plates are located in the four directions of front, back, left and right to collect images.

9. The method according to claim 1, characterized in that The at least two cameras include two black-and-white cameras, and the three-dimensional scanning device further includes a texture camera. After obtaining a calibration extrinsic parameter between the two black-and-white cameras, the method further includes: Determining the three-dimensional coordinates of a first marker point and a second marker point in the calibration device based on the calibration extrinsic parameter; For any one of the two black-and-white cameras, using the black-and-white camera and the texture camera to respectively collect images of the calibration device; The calibration extrinsic parameters of the black-and-white camera and the texture camera are determined based on the collected multiple frames of calibration images and the three-dimensional coordinates of the first marker point and the second marker point.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed, the method according to any one of claims 1 to 9 is implemented.

11. An electronic device, characterized in that: The electronic device includes a processor, a memory, and computer instructions stored in the memory and executable by the processor. When the processor executes the computer instructions, the method according to any one of claims 1 to 9 is implemented.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 9.

13. A calibration device, characterized in that: The calibration device includes a marker and at least two calibration plates, the marker includes multiple first marker points, the real three-dimensional coordinates of each of the multiple first marker points have been predetermined and stored, each first marker point corresponds to an identifier, which is used to uniquely identify the first marker point, and the identifier is stored in association with the real three-dimensional coordinates of the first marker point, and each calibration plate includes multiple second marker points whose relative position relationships are unknown; wherein, when using the calibration device to calibrate the equipment, the at least two calibration plates are placed near the marker to combine to obtain a large-sized combined calibration plate.

14. The calibration device according to claim 13, characterized in that: The plurality of first marker points in the marker can be distinguished by visual information; and / or The marker is a long strip-shaped marker. When the calibration device is used to calibrate the equipment, the at least two calibration plates are symmetrically placed along the long side of the marker.

15. The calibration device according to claim 14, characterized in that: The number of the calibration plates is 6, and 3 calibration plates are placed on each side of the marker.

16. The calibration device according to claim 14, characterized in that: The visual information includes one or more of the following: the size of the first marker point, the shape of the first marker point, the color of the first marker point, the arrangement of the first marker point, and marking information located around the first marker point; and / or The plurality of first marking points in the marker are divided into a plurality of groups, and the arrangement manners of the first marking points in the plurality of groups are different.

Images