A calibration method, system and device

Abstract

The present application provides a calibration method, system and device. By adding an auxiliary camera, since the frame and resolution of the auxiliary camera are larger than the frame and resolution of the scanner camera, a wider picture can be captured during acquisition, thereby smoothing the difference in the number of feature corner points extracted from different positions on the calibration plate image, and reducing the number of calibration plate images collected. At the same time, uniform and sufficient feature points can be extracted, so that the calibration result of the projection device is more accurate, the number of calibrations is reduced, and the calibration efficiency is improved. Moreover, when performing multi-camera joint calibration, since the multi-camera constraints can provide more stable constraints, the error is smaller during the adjustment calculation, making the camera calibration more accurate. At the same time, the more feature points there are, the smaller the error calculated during calibration, and the higher the calibration accuracy.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a calibration method, system and device. Background Art

[0002] The primary task of computer vision is to obtain the corresponding information of objects in the real three-dimensional world through the captured image information. Therefore, it is particularly important to establish a geometric model in the process of mapping objects from the three-dimensional world to the camera imaging plane.

[0003] For 3D scanners with coded structured light, the structure is mainly composed of a camera and a projection device. Its working principle is to project the structured light code from the projection device onto the object, which is then captured by the sensor to form an image. Then, the distance between each point of the object and the camera plane is calculated based on the distortion of the coded pattern and the internal and external parameters of the sensor. Therefore, the most critical part of this process is to calibrate the sensor and the projection device to obtain their internal and external parameters.

[0004] When calibrating the coded structured light 3D scanner, commonly used calibration methods include Zhang Zhengyou's planar calibration method, Tsai's two-step calibration method, etc. These calibration methods require taking more calibration images, are not efficient, and have relatively large errors during calibration. Summary of the invention

[0005] The purpose of the present invention is to provide a calibration method, system and device to solve the problem of how to calibrate a coded structured light 3D scanner efficiently and accurately.

[0006] In order to achieve the above-mentioned object of the invention, the present invention provides the following technical solutions:

[0007] In a first aspect, a calibration method is provided, which is applied to a calibration system, wherein the calibration system comprises: a terminal, a 3D scanner, and an auxiliary camera, wherein the 3D scanner comprises a scanner camera and a projection device, wherein the frame of the auxiliary camera is larger than the frame of the scanner camera, and the resolution of the auxiliary camera is larger than the resolution of the scanner camera;

[0008] The method comprises:

[0009] The first acquisition step: the terminal acquires the calibration plate image captured by the auxiliary camera and the scanner camera;

[0010] The first extraction step: the terminal extracts the characteristic corner points of the calibration plate image;

[0011] Scanner camera calibration step: the terminal calculates a first internal parameter and a first external parameter according to the feature corner point, wherein the first internal parameter is an internal parameter of the scanner camera, and the first external parameter is an external parameter of the scanner camera;

[0012] The second acquisition step: the terminal acquires the projection image captured by the auxiliary camera and the scanner camera, and the projection image is an image formed by the projection device projected on the whiteboard;

[0013] Second extraction step: the terminal extracts feature points of the projection image;

[0014] Projection device calibration step: the terminal calculates a second internal parameter and a second external parameter according to the first internal parameter, the first external parameter and the feature point, wherein the second internal parameter is an internal parameter of the projection device, and the second external parameter is an external parameter of the projection device;

[0015] Output step: The terminal outputs the calibration file according to the first internal parameter, the first external parameter, the second internal parameter, and the second external parameter.

[0016] In a second aspect, a calibration system is provided, the calibration system comprising: a terminal, a 3D scanner, and an auxiliary camera, the 3D scanner comprising a scanner camera and a projection device, the frame of the auxiliary camera is larger than the frame of the scanner camera, and the resolution of the auxiliary camera is larger than the resolution of the scanner camera;

[0017] The scanner camera is used to collect the calibration plate image and the projection image;

[0018] The auxiliary camera is used to collect the calibration plate image and the projection image;

[0019] The projection device is used to project the coding pattern onto the whiteboard;

[0020] The terminal is used to perform the following steps:

[0021] The first acquisition step: acquiring the calibration plate image captured by the auxiliary camera and the scanner camera;

[0022] The first extraction step: extracting characteristic corner points of the calibration plate image;

[0023] Scanner camera calibration step: calculating a first internal parameter and a first external parameter according to the feature corner point, wherein the first internal parameter is an internal parameter of the scanner camera, and the first external parameter is an external parameter of the scanner camera;

[0024] The second acquisition step: acquiring the projection image captured by the auxiliary camera and the scanner camera, wherein the projection image is an image formed by the projection device projecting onto the whiteboard;

[0025] The second extraction step: extracting feature points of the projection image;

[0026] A projection device calibration step: calculating a second internal parameter and a second external parameter according to the first internal parameter, the first external parameter and the feature point, wherein the second internal parameter is an internal parameter of the projection device, and the second external parameter is an external parameter of the projection device;

[0027] Output step: output the calibration file according to the first internal parameter, the first external parameter, the second internal parameter, and the second external parameter.

[0028] According to a third aspect, a computer device is provided, comprising:

[0029] A memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the following steps:

[0030] The first acquisition step: acquiring the calibration plate image captured by the auxiliary camera and the scanner camera;

[0031] The first extraction step: extracting characteristic corner points of the calibration plate image;

[0032] Scanner camera calibration step: calculating a first internal parameter and a first external parameter according to the feature corner point, wherein the first internal parameter is an internal parameter of the scanner camera, and the first external parameter is an external parameter of the scanner camera;

[0033] The second acquisition step: acquiring the projection image captured by the auxiliary camera and the scanner camera, wherein the projection image is an image formed by the projection device projecting onto the whiteboard;

[0034] The second extraction step: extracting feature points of the projection image;

[0035] A projection device calibration step: calculating a second internal parameter and a second external parameter according to the first internal parameter, the first external parameter and the feature point, wherein the second internal parameter is an internal parameter of the projection device, and the second external parameter is an external parameter of the projection device;

[0036] Output step: output the calibration file according to the first internal parameter, the first external parameter, the second internal parameter, and the second external parameter.

[0037] The calibration method, system and device described above first obtain the calibration plate image captured by the auxiliary camera and the scanner camera, thereby extracting the characteristic corner points of the calibration plate image, and then calculating the first internal parameter and the first external parameter based on the characteristic corner points. With the addition of the auxiliary camera, since the frame and resolution of the auxiliary camera are larger than the frame and resolution of the scanner camera, a wider picture can be captured during acquisition, thereby smoothing the difference in the number of characteristic corner points extracted from different positions on the calibration plate image, thereby reducing the number of calibration plate images collected and improving the calibration efficiency. Moreover, when performing multi-camera joint calibration, since the multi-camera constraints can provide more stable constraints, the error is smaller during the adjustment calculation, making the camera calibration more accurate.

[0038] Secondly, by acquiring the projection image captured by the auxiliary camera and the scanner camera, the feature points of the projection image are extracted, and then the second internal parameter and the second external parameter are calculated according to the first internal parameter, the first external parameter and the feature point. With the addition of the auxiliary camera, since the frame and resolution of the auxiliary camera are larger than those of the scanner camera, a wider picture can be captured during the acquisition, and uniform and sufficient feature points can be extracted, so that the calibration result of the projection device is more accurate, the number of calibration times is reduced, and the calibration efficiency is improved. Moreover, the more feature points there are, the smaller the error calculated during calibration, and the higher the calibration accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] in:

[0041] Figure 1 is a flow chart of a calibration method in one embodiment;

[0042] Figure 2 is a system block diagram of a calibration system in one embodiment;

[0043] Figure 3 A schematic diagram of a picture frame in an embodiment;

[0044] Figure 4 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0045] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0046] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0047] It should be noted that the terms "include", "comprises" and "have" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, terminal, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices. In the claims, specification and drawings of the present application, relational terms such as "first" and "second" are merely used to distinguish one entity / operation / object from another entity / operation / object, and do not necessarily require or imply any such actual relationship or order between these entities / operations / objects.

[0048] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0049] The algorithm steps for building a three-dimensional model through images are: camera restoration → model restoration → stereo correspondence. Camera restoration refers to solving the internal parameters of the camera used when the image was taken and the external parameters of the camera. For a 3D scanner with coded structured light, its structure is mainly composed of a camera and a projection device. Its working principle is mainly to project the structured light code from the projection device onto the object to be acquired by the sensor to form an image, and then calculate the distance of each point of the object from the camera plane based on the distortion of the coded pattern and the internal and external parameters of the sensor. Therefore, the most critical part of this process is to calibrate the sensor and the projection device to obtain its internal and external parameters.

[0050] When calibrating, the traditional calibration method can extract relatively few and uneven feature corner points and feature points from the calibration plate images and projection images captured by the camera. Therefore, more calibration plate images and projection images need to be collected. As a result, the efficiency is low, and the calculation accuracy is not high enough, requiring multiple calibrations.

[0051] like Figure 1 As shown, a calibration method is proposed, which is applied to a calibration system, wherein the calibration system comprises: a terminal 201, a 3D scanner, and an auxiliary camera 202, wherein the 3D scanner comprises a scanner camera 203 and a projection device 204, wherein the frame of the auxiliary camera is larger than the frame of the scanner camera, and the resolution of the auxiliary camera is larger than the resolution of the scanner camera;

[0052] The method comprises:

[0053] First acquisition step 101: the terminal 201 acquires the calibration plate image captured by the auxiliary camera 202 and the scanner camera 203 .

[0054] Before the auxiliary camera 202 and the scanner camera 203 collect the calibration plate image, the equipment needs to be calibrated, specifically: first fix the position of the 3D scanner and the auxiliary camera 202, so that the auxiliary camera 202, the scanner camera 203, and the projection device 204 form a triangular position relationship, and adjust the lens focal length of the auxiliary camera 202, the scanner camera 203, and the optical projection device 204 so that their images at the focal length position are the clearest. When the equipment is calibrated, the auxiliary camera 202 and the scanner camera 203 start to collect the calibration plate image. Among them, the calibration plate used for the calibration of the auxiliary camera 202 and the scanner camera 203 can be made of glass material, and then a customized pattern is printed on the glass plate. The auxiliary camera 202 and the scanner camera 203 shoot a flat plate with a fixed spacing pattern array, and after calculation by the calibration algorithm, the geometric model of the auxiliary camera 202 and the scanner camera 203 can be obtained, thereby obtaining high-precision measurement and reconstruction results, and the flat plate with a fixed spacing pattern array is the calibration plate. Within the depth of field range of the auxiliary camera 202 and the scanner camera 203 (depth of field refers to the distance range before and after the object measured by the imaging that can obtain a clear image at the front of the camera lens or other imager. After focusing is completed, a clear image can be formed in the range before and after the focus. This distance range is called the depth of field), the calibration plate with different postures is imaged from near to far. The meaning of being within the depth of field is to make the captured image clear enough to ensure that the auxiliary camera 202 and the scanner camera 203 can extract enough feature corner points each time they capture the image. After the auxiliary camera 202 and the scanner camera 203 capture the calibration plate image, the terminal 201 obtains the calibration plate image captured by the auxiliary camera 202 and the scanner camera 203. The terminal 201 includes a computer or a tablet computer.

[0055] First extraction step 102: the terminal 201 extracts characteristic corner points of the calibration plate image.

[0056] After the terminal 201 obtains the calibration plate image, the characteristic corner points in the calibration plate image are extracted. The characteristic corner points are the intersection points between contours. For the same scene, even if the viewing angle changes, the characteristics of the point are usually stable; the pixel points in the area near the point have large changes in both the gradient direction and the gradient amplitude.

[0057] Scanner camera calibration step 103 : the terminal 201 calculates a first internal parameter and a first external parameter according to the feature corner point, wherein the first internal parameter is an internal parameter of the scanner camera 203 , and the first external parameter is an external parameter of the scanner camera 203 .

[0058] Among them, Figure 3 As shown, 301 is the frame of the auxiliary camera 202, and 302 is the frame of the scanner camera 203. The frame of the auxiliary camera 202 is larger than that of the scanner camera 203. The frame refers to the size of the image sensor area in the camera. The larger the frame, that is, the larger the area of ​​the sensor, the larger the imaging area, the more pixels and the finer the image. In short, the better the image quality. The resolution of the auxiliary camera 202 is greater than the resolution of the scanner camera 203. Therefore, the number of feature corner points extracted from the calibration plate image collected by the auxiliary camera 202 is greater than the number of feature corner points extracted from the calibration plate image collected by the scanner camera 203, and the feature corner points extracted from the calibration plate image collected by the auxiliary camera 202 are more accurate and uniform. The terminal 201 calculates the first internal parameter and the first external parameter according to the feature corner point. Specifically, the terminal 201 obtains the image coordinates of the feature corner point; the terminal 201 calculates the coordinates of the same name points of the scanner camera 203 and the auxiliary camera 202 according to the image coordinates; the terminal 201 establishes a mapping relationship between the image coordinates of the feature corner point and the coordinates of the same name points; the terminal 201 calculates the first internal parameter and the first external parameter according to the mapping relationship. The internal parameters refer to the focal length of the sensor lens, the offset of the optical center, and the distortion, and the external parameters include the sensor rotation parameter and the translation parameter, indicating their relative positions. In camera calibration, the calibration plate is used as a fixed reference coordinate system. The calibration plate is composed of known points arranged in a certain pattern. The distances between points in the horizontal and vertical directions are known, and the three-dimensional coordinates of the points are known. The points are located at the intersection of the grid on the calibration plate, and corner features are presented on the image. Then, the coordinates of the same-name points of the scanner camera 203 and the auxiliary camera 202 are calculated through the stereo matching algorithm through the image coordinates of these feature corner points, so as to establish a mapping relationship between the two, and the internal and external parameters of the scanner camera 203 are calculated through the relevant theoretical methods of stereo matching. By adding the auxiliary camera 202 with a larger frame and higher resolution to help the scanner camera 203 calibrate, the difference in the number of feature corner points extracted from different positions on the calibration plate image can be smoothed, and the extracted feature corner points are more accurate, thereby reducing the number of calibration plate images collected and improving calibration efficiency.

[0059] After calibrating a single camera, multiple cameras are calibrated together and an adjustment is performed. The error converges through iterative calculations, and the error becomes smaller with each iteration. Therefore, when multiple cameras are calibrated together by adding an auxiliary camera 202 with a larger image, the error is smaller during the adjustment calculation because the multi-camera constraint can provide a more stable constraint, so that the scanner camera 203 has a higher accuracy during calibration.

[0060] Second acquisition step 104: the terminal 201 acquires the projection image captured by the auxiliary camera 202 and the scanner camera 203 , where the projection image is an image formed by the projection device 204 projecting onto the whiteboard.

[0061] Among them, the 3D scanner used in this application is mainly a coded structured light 3D scanner, and its structure is mainly composed of a camera and a projection device 204. The structured light code is projected from the projection device 204 onto the object and captured by the sensor to form a picture, and then the distance of each point of the object from the camera plane is calculated according to the distortion of the coding pattern and the internal and external parameters of the sensor. The auxiliary camera 202 and the scanner camera 203 capture the image formed by the projection device 204 projected on the whiteboard, and use the whiteboard to present the pattern projected by the optical projector. During the acquisition, it is necessary to collect the projection patterns on the whiteboard in different postures from near to far within the depth of field of the camera, and the terminal 201 obtains the projection patterns collected by the auxiliary camera 202 and the scanner camera 203.

[0062] Second extraction step 105: the terminal 201 extracts feature points of the projection image.

[0063] After the terminal 201 obtains the projection pattern captured by the auxiliary camera 202 and the scanner camera 203, the feature points of the projection image are extracted. The feature points refer to the points where the gray value of the image changes dramatically or the points with large curvature on the edge of the image (i.e., the intersection of two edges).

[0064] Projection device calibration step 106: the terminal 201 calculates a second internal parameter and a second external parameter according to the first internal parameter, the first external parameter and the feature point, wherein the second internal parameter is an internal parameter of the projection device 204 and the second external parameter is an external parameter of the projection device 204.

[0065] The terminal 201 calculates the second internal parameter and the second external parameter according to the first internal parameter, the first external parameter and the feature point, which specifically includes: the terminal 201 obtains the image coordinates of the feature point; the terminal 201 calculates the homography matrix according to the first internal parameter, the first external parameter and the feature point; the terminal 201 calculates the three-dimensional coordinates of the feature point according to the homography matrix; the terminal 201 calculates the second internal parameter and the second external parameter according to the image coordinates of the feature point and the three-dimensional coordinates of the feature point. Since the first internal parameter and the first external parameter of the scanner camera 203 are obtained in the scanner camera calibration step 103, the feature point is extracted in the second extraction step 105, and the image coordinates of the feature point are known. The homography matrix can be solved by the first internal parameter, the first external parameter and the feature point through the sampling consistency algorithm, and the three-dimensional coordinates of the above feature point are calculated by the homography matrix. Based on the feature point coordinates of the object plane and the three-dimensional coordinates of the corresponding space point, the internal parameters and external parameters of the projection device 204 are calculated. In the sampling consistency algorithm, the more the number of feature points, the better the effect and the smaller the error. Since the frame and resolution of the auxiliary camera 202 are larger than those of the scanner camera 203, the number of feature points extracted from the projection pattern captured by the auxiliary camera 202 is larger than that of the projection pattern captured by the scanner camera 203. The smaller the calibration error of the projection device 204, the more accurate the calibration result. Moreover, evenly and sufficiently many feature points are extracted, so that the calibration result of the projection device 204 is more accurate, the calibration times are reduced, and the calibration efficiency is improved.

[0066] Output step 107: the terminal 201 outputs a calibration file according to the first internal parameter, the first external parameter, the second internal parameter, and the second external parameter.

[0067] When the internal parameters and external parameters of the scanner camera 203 and the projection device 204 are calculated, the internal parameters and external parameters of the scanner camera 203 and the projection device 204 are written into the calibration file, and the calibration is considered to be completed.

[0068] The above calibration method first obtains the calibration plate image captured by the auxiliary camera 202 and the scanner camera 203, thereby extracting the characteristic corner points of the calibration plate image, and then calculating the first internal parameter and the first external parameter based on the characteristic corner points. With the addition of the auxiliary camera 202, since the frame and resolution of the auxiliary camera 202 are larger than the frame and resolution of the scanner camera 203, a wider picture can be captured during acquisition, thereby smoothing the difference in the number of characteristic corner points extracted from different positions on the calibration plate image, thereby reducing the number of calibration plate images collected and improving the calibration efficiency. Moreover, when performing multi-camera joint calibration, since the multi-camera constraints can provide more stable constraints, the error is smaller during the adjustment calculation, making the camera calibration more accurate.

[0069] Secondly, by acquiring the projection image captured by the auxiliary camera 202 and the scanner camera 203, the feature points of the projection image are extracted, and then the second internal parameter and the second external parameter are calculated according to the first internal parameter, the first external parameter and the feature point. With the addition of the auxiliary camera 202, since the frame and resolution of the auxiliary camera 202 are larger than the frame and resolution of the scanner camera 203, a wider picture can be captured during the acquisition, and uniform and sufficient feature points can be extracted, so that the calibration result of the projection device 204 is more accurate, the number of calibration times is reduced, and the calibration efficiency is improved. And the more feature points there are, the smaller the error calculated during calibration, and the higher the accuracy during calibration.

[0070] In one embodiment, the terminal 201 calculates the first internal parameter and the first external parameter according to the feature corner point, including: the terminal 201 obtains the image coordinates of the feature corner point; the terminal 201 calculates the coordinates of the same-name points of the scanner camera 203 and the auxiliary camera 202 according to the image coordinates; the terminal 201 establishes a mapping relationship between the image coordinates of the feature corner point and the coordinates of the same-name points; the terminal 201 calculates the first internal parameter and the first external parameter according to the mapping relationship.

[0071] Specifically, the terminal 201 calculates the first internal parameter and the first external parameter according to the feature corner point: the terminal 201 obtains the image coordinates of the feature corner point; the terminal 201 calculates the coordinates of the same name points of the scanner camera 203 and the auxiliary camera 202 according to the image coordinates; the terminal 201 establishes a mapping relationship between the image coordinates of the feature corner point and the coordinates of the same name points; the terminal 201 calculates the first internal parameter and the first external parameter according to the mapping relationship. The internal parameters refer to the focal length of the sensor lens, the offset of the optical center, and the distortion, and the external parameters include the sensor rotation parameters and the translation parameters, indicating their relative positions. In camera calibration, the calibration plate is used as a fixed reference coordinate system. The calibration plate is composed of known points arranged in a certain pattern. The distances between points in the horizontal and vertical directions are known, and the three-dimensional coordinates of the points are known. The points are located at the intersection of the grid on the calibration plate, and corner features are presented on the image. Then, the coordinates of the same-name points of the scanner camera 203 and the auxiliary camera 202 are calculated through the stereo matching algorithm through the image coordinates of these feature corner points, so as to establish a mapping relationship between the two, and the internal and external parameters of the scanner camera 203 are calculated through the relevant theoretical methods of stereo matching. By adding the auxiliary camera 202 with a larger frame and higher resolution to help the scanner camera 203 calibrate, the difference in the number of feature corner points extracted from different positions on the calibration plate image can be smoothed, and the extracted feature corner points are more accurate, thereby reducing the number of calibration plate images collected and improving calibration efficiency.

[0072] In one embodiment, after the terminal 201 calculates the first internal parameter and the first external parameter according to the coordinates of the same-name points, it also includes: the terminal 201 calculates the first calibration error of the scanner camera 203; the terminal 201 determines whether the first calibration error is lower than a first preset value; if so, it is deemed that the calibration is qualified and jumps to the second acquisition step; if not, it is deemed that the calibration is unqualified and jumps to the first acquisition step 101.

[0073] Among them, after the terminal 201 completes the calibration of the scanner camera 203, it is still necessary to check whether the calibration result is accurate. First, the first calibration error is calculated, and the coordinates of the three-dimensional points corresponding to the collected two-dimensional calibration plate image are calculated by back projection. Since the three-dimensional coordinates of the points on the calibration plate are known, the coordinates of the three-dimensional points corresponding to the collected two-dimensional image calculated by back projection and the three-dimensional coordinates of the points on the calibration plate are compared. The relationship between this error and the first preset value is judged. The first preset value can be designed by itself, for example, set to 0.1. When the error is less than the first preset value, it is considered that the calibration is qualified; when the error is not less than the first preset value, it is considered that the calibration is unqualified. A quantitative evaluation standard is established to verify whether the calibration result is qualified, and the next step is performed according to the judgment result to ensure that the calibration result is qualified.

[0074] In one embodiment, the terminal 201 calculates the first calibration error of the scanner camera 203, including: the terminal 201 obtains the first three-dimensional coordinates of the feature corner point; the terminal 201 calculates the second three-dimensional coordinates of the feature corner point according to the image coordinates of the feature corner point; the terminal 201 calculates the error between the first three-dimensional coordinates and the second three-dimensional coordinates.

[0075] Among them, the first calibration error is calculated first, and the coordinates of the three-dimensional points corresponding to the collected two-dimensional calibration plate image are calculated by back projection. Since the image coordinates of the calibration plate image and the three-dimensional coordinates of the points on the calibration plate are known, the error between the coordinates of the three-dimensional points corresponding to the collected two-dimensional image calculated by back projection and the three-dimensional coordinates of the points on the calibration plate is obtained by comparing.

[0076] In one embodiment, the terminal 201 calculates the second internal parameter and the second external parameter based on the first internal parameter, the first external parameter and the feature point, including: the terminal 201 obtains the image coordinates of the feature point; the terminal 201 calculates the homography matrix based on the first internal parameter, the first external parameter and the feature point; the terminal 201 calculates the three-dimensional coordinates of the feature point based on the homography matrix; the terminal 201 calculates the second internal parameter and the second external parameter based on the image coordinates of the feature point and the three-dimensional coordinates of the feature point.

[0077] The terminal 201 calculates the second internal parameter and the second external parameter according to the first internal parameter, the first external parameter and the feature point, which specifically includes: the terminal 201 obtains the image coordinates of the feature point; the terminal 201 calculates the homography matrix according to the first internal parameter, the first external parameter and the feature point; the terminal 201 calculates the three-dimensional coordinates of the feature point according to the homography matrix; the terminal 201 calculates the second internal parameter and the second external parameter according to the image coordinates of the feature point and the three-dimensional coordinates of the feature point. Since the first internal parameter and the first external parameter of the scanner camera 203 are obtained in the scanner camera calibration step 103, the feature point is extracted in the second extraction step 105, and the image coordinates of the feature point are known. The homography matrix can be solved by the first internal parameter, the first external parameter and the feature point through the sampling consistency algorithm, and the three-dimensional coordinates of the above feature point are calculated by the homography matrix. Based on the feature point coordinates of the object plane and the three-dimensional coordinates of the corresponding space point, the internal parameters and external parameters of the projection device 204 are calculated. In the sampling consistency algorithm, the more the number of feature points, the better the effect and the smaller the error. Since the frame and resolution of the auxiliary camera 202 are larger than those of the scanner camera 203, the number of feature points extracted from the projection pattern captured by the auxiliary camera 202 is larger than that of the projection pattern captured by the scanner camera 203. The smaller the calibration error of the projection device 204, the more accurate the calibration result. Moreover, evenly and sufficiently many feature points are extracted, so that the calibration result of the projection device 204 is more accurate, the calibration times are reduced, and the calibration efficiency is improved.

[0078] In one embodiment, after the terminal 201 calculates the second internal parameter and the second external parameter according to the first internal parameter, the first external parameter and the feature point, the method further includes:

[0079] Terminal 201 calculates the second calibration error of the projection device 204; terminal 201 determines whether the second calibration error is lower than a second preset value; if so, it is considered that the calibration is qualified and jumps to output step 107; if not, it is considered that the calibration is unqualified and jumps to the second acquisition step.

[0080] Among them, after the terminal 201 completes the calibration of the projection device 204, it is still necessary to check whether the calibration result is accurate. First, the second calibration error is calculated, and the coordinates of the three-dimensional points corresponding to the collected two-dimensional projection image are calculated by back projection. Since the three-dimensional coordinates of the points on the whiteboard are known, the coordinates of the three-dimensional points corresponding to the collected two-dimensional image calculated by back projection and the three-dimensional coordinates of the points on the whiteboard are compared. The relationship between this error and the second preset value is judged. The second preset value can be designed by itself, for example, set to 0.1. When the error is less than the second preset value, it is considered that the calibration is qualified; when the error is not less than the second preset value, it is considered that the calibration is unqualified. A quantitative evaluation standard is established to verify whether the calibration result is qualified, and the next step is performed according to the judgment result to ensure that the calibration result is qualified.

[0081] In one embodiment, after the output step 107, the method further includes: a judgment step 108: the terminal 201 establishes a quantitative evaluation standard, and judges whether the calibration file is qualified according to the quantitative evaluation standard.

[0082] Among them, after the terminal 201 outputs the calibration file, it is also necessary to check whether the calibration file is accurate. Specifically, it includes: the terminal 201 obtains the three-dimensional space point corresponding to the feature point; the terminal 201 constructs a triangular mesh according to the three-dimensional space point; the terminal 201 obtains the complete number of the triangular mesh; the terminal 201 determines whether the complete number of the triangular mesh is greater than a third preset value. The third preset value can be designed by itself, for example, set to 10. By establishing a quantitative evaluation standard to verify whether the calibration file is qualified, it is guaranteed that the calibration result is qualified.

[0083] In one embodiment, the quantitative evaluation criteria include: terminal 201 obtains the three-dimensional space point corresponding to the feature point; terminal 201 constructs a triangular mesh based on the three-dimensional space point; terminal 201 obtains the complete number of the triangular mesh; terminal 201 determines whether the complete number of the triangular mesh is greater than a third preset value.

[0084] The third preset value can be designed by oneself, for example, set to 10. A quantitative evaluation standard is established to verify whether the calibration file is qualified, thereby ensuring that the calibration result is qualified.

[0085] like Figure 2 As shown, a calibration system is proposed, the calibration system includes: a terminal 201, a 3D scanner, an auxiliary camera 202, the 3D scanner includes a scanner camera 203, a projection device 204, the frame of the auxiliary camera 202 is larger than the frame of the scanner camera 203, and the resolution of the auxiliary camera 202 is larger than the resolution of the scanner camera 203;

[0086] The scanner camera 203 is used to collect the calibration plate image;

[0087] The auxiliary camera 202 is used to collect the calibration plate image;

[0088] The projection device 204 is used to project the coding pattern onto the whiteboard;

[0089] The terminal 201 is used to perform the following steps:

[0090] First acquisition step 101: acquiring the calibration plate image captured by the auxiliary camera 202 and the scanner camera 203;

[0091] First extraction step 102: extracting characteristic corner points of the calibration plate image;

[0092] Scanner camera calibration step 103: calculating a first internal parameter and a first external parameter according to the feature corner point, wherein the first internal parameter is an internal parameter of the scanner camera 203 and the first external parameter is an external parameter of the scanner camera 203;

[0093] Second acquisition step 104: acquiring the projection image captured by the auxiliary camera 202 and the scanner camera 203, wherein the projection image is an image formed by the projection device 204 projecting onto the whiteboard;

[0094] Second extraction step 105: extracting feature points of the projection image;

[0095] Projection device calibration step 106: calculating a second internal parameter and a second external parameter according to the first internal parameter, the first external parameter and the feature point, wherein the second internal parameter is an internal parameter of the projection device 204 and the second external parameter is an external parameter of the projection device 204;

[0096] Output step 107: output a calibration file according to the first internal parameter, the first external parameter, the second internal parameter, and the second external parameter.

[0097] The above calibration system first obtains the calibration plate image captured by the auxiliary camera 202 and the scanner camera 203, thereby extracting the characteristic corner points of the calibration plate image, and then calculating the first internal parameter and the first external parameter based on the characteristic corner points. With the addition of the auxiliary camera 202, since the frame and resolution of the auxiliary camera 202 are larger than the frame and resolution of the scanner camera 203, a wider picture can be captured during acquisition, thereby smoothing the difference in the number of characteristic corner points extracted from different positions on the calibration plate image, thereby reducing the number of calibration plate images collected and improving the calibration efficiency. Moreover, when performing multi-camera joint calibration, since the multi-camera constraints can provide more stable constraints, the error is smaller during the adjustment calculation, making the camera calibration more accurate.

[0098] Secondly, by acquiring the projection image captured by the auxiliary camera 202 and the scanner camera 203, the feature points of the projection image are extracted, and then the second internal parameter and the second external parameter are calculated according to the first internal parameter, the first external parameter and the feature point. With the addition of the auxiliary camera 202, since the frame and resolution of the auxiliary camera 202 are larger than the frame and resolution of the scanner camera 203, a wider picture can be captured during the acquisition, and uniform and sufficient feature points can be extracted, so that the calibration result of the projection device 204 is more accurate, the number of calibration times is reduced, and the calibration efficiency is improved. And the more feature points there are, the smaller the error calculated during calibration, and the higher the accuracy during calibration.

[0099] In one embodiment, the terminal 201 calculates the first internal parameter and the first external parameter according to the feature corner point, including: the terminal 201 obtains the image coordinates of the feature corner point; the terminal 201 calculates the coordinates of the same-name points of the scanner camera 203 and the auxiliary camera 202 according to the image coordinates; the terminal 201 establishes a mapping relationship between the image coordinates of the feature corner point and the coordinates of the same-name points; the terminal 201 calculates the first internal parameter and the first external parameter according to the mapping relationship.

[0100] In one embodiment, after the terminal 201 calculates the first internal parameter and the first external parameter according to the coordinates of the same-name points, it also includes: the terminal 201 calculates the first calibration error of the scanner camera 203; the terminal 201 determines whether the first calibration error is lower than a first preset value; if so, it is deemed that the calibration is qualified and jumps to the second acquisition step; if not, it is deemed that the calibration is unqualified and jumps to the first acquisition step 101.

[0101] In one embodiment, the terminal 201 calculates the first calibration error of the scanner camera 203, including: the terminal 201 obtains the first three-dimensional coordinates of the feature corner point; the terminal 201 calculates the second three-dimensional coordinates of the feature corner point according to the image coordinates of the feature corner point; the terminal 201 calculates the error between the first three-dimensional coordinates and the second three-dimensional coordinates.

[0102] In one embodiment, the terminal 201 calculates the second internal parameter and the second external parameter based on the first internal parameter, the first external parameter and the feature point, including: the terminal 201 obtains the image coordinates of the feature point; the terminal 201 calculates the homography matrix based on the first internal parameter, the first external parameter and the feature point; the terminal 201 calculates the three-dimensional coordinates of the feature point based on the homography matrix; the terminal 201 calculates the second internal parameter and the second external parameter based on the image coordinates of the feature point and the three-dimensional coordinates of the feature point.

[0103] In one embodiment, after the terminal 201 calculates the second internal parameter and the second external parameter according to the first internal parameter, the first external parameter and the feature point, it also includes: the terminal 201 calculates the second calibration error of the projection device 204; the terminal 201 determines whether the second calibration error is lower than a second preset value; if so, it is regarded as qualified calibration and jumps to the output step 107; if not, it is regarded as unqualified calibration and jumps to the second acquisition step.

[0104] In one embodiment, after the output step 107, the method further includes: a judgment step 108: the terminal 201 establishes a quantitative evaluation standard, and judges whether the calibration file is qualified according to the quantitative evaluation standard.

[0105] In one embodiment, the quantitative evaluation criteria include: terminal 201 obtains the three-dimensional space point corresponding to the feature point; terminal 201 constructs a triangular mesh based on the three-dimensional space point; terminal 201 obtains the complete number of the triangular mesh; terminal 201 determines whether the complete number of the triangular mesh is greater than a third preset value.

[0106] like Figure 4 As shown, the computer device includes a processor, a memory and a network interface connected via a terminal 201 bus. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device has a storage operation terminal 201 and may also store a computer program. When the computer program is executed by the processor, the processor can implement the above-mentioned calibration method. The internal memory may also store a computer program. When the computer program is executed by the processor, the processor can execute the above-mentioned calibration method. Those skilled in the art can understand that Figure 4 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the device to which the scheme of the present application is applied. The specific device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0107] In one embodiment, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the processor executes the steps of the above-mentioned calibration method.

[0108] It is understandable that the above-mentioned calibration method, device and storage medium belong to a general inventive concept, and the embodiments are applicable to each other.

[0109] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0110] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0111] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A calibration method, applied to a calibration system, characterized in that: The calibration system comprises: a terminal, a 3D scanner, and an auxiliary camera, wherein the 3D scanner comprises a scanner camera and a projection device, the frame of the auxiliary camera is larger than the frame of the scanner camera, and the resolution of the auxiliary camera is larger than the resolution of the scanner camera; The method comprises: The first acquisition step: the terminal acquires the calibration plate image captured by the auxiliary camera and the scanner camera; The first extraction step: the terminal extracts the characteristic corner points of the calibration plate image; Scanner camera calibration step: the terminal calculates a first internal parameter and a first external parameter according to the feature corner point, wherein the first internal parameter is an internal parameter of the scanner camera, and the first external parameter is an external parameter of the scanner camera; The second acquisition step: the terminal acquires the projection image captured by the auxiliary camera and the scanner camera, and the projection image is an image formed by the projection device projected on the whiteboard; Second extraction step: the terminal extracts feature points of the projection image; Projection device calibration step: the terminal calculates a second internal parameter and a second external parameter according to the first internal parameter, the first external parameter and the feature point, wherein the second internal parameter is an internal parameter of the projection device, and the second external parameter is an external parameter of the projection device; Output step: The terminal outputs the calibration file according to the first internal parameter, the first external parameter, the second internal parameter, and the second external parameter.

2. The calibration method according to claim 1, characterized in that: The terminal calculates a first internal parameter and a first external parameter according to the feature corner point, including: The terminal obtains the image coordinates of the feature corner points; The terminal calculates the coordinates of the same-name points of the scanner camera and the auxiliary camera according to the image coordinates; The terminal establishes a mapping relationship between the image coordinates of the feature corner points and the coordinates of the points with the same name; The terminal calculates the first internal parameter and the first external parameter according to the mapping relationship.

3. The calibration method according to claim 2, characterized in that: After the terminal calculates the first internal parameter and the first external parameter according to the mapping relationship, the method further includes: The terminal calculates a first calibration error of the scanner camera; The terminal determines whether the first calibration error is lower than a first preset value; If yes, it is considered as qualified and jumps to the second acquisition step; If not, it is considered as calibration failure and jumps to the first acquisition step.

4. The calibration method according to claim 3, characterized in that: The terminal calculates a first calibration error of the scanner camera, including: The terminal obtains the first three-dimensional coordinates of the feature corner point; The terminal calculates the second three-dimensional coordinates of the feature corner point according to the image coordinates of the feature corner point; The terminal calculates an error between the first three-dimensional coordinate and the second three-dimensional coordinate.

5. The calibration method according to claim 1, characterized in that: The terminal calculates a second internal parameter and a second external parameter according to the first internal parameter, the first external parameter and the feature point, including: The terminal obtains the image coordinates of the feature points; The terminal calculates a homography matrix according to the first internal parameter, the first external parameter and the feature point; The terminal calculates the three-dimensional coordinates of the feature points according to the homography matrix; The terminal calculates the second internal parameter and the second external parameter according to the image coordinates of the feature point and the three-dimensional coordinates of the feature point.

6. The calibration method according to claim 5, characterized in that: After the terminal calculates the second internal parameter and the second external parameter according to the first internal parameter, the first external parameter and the feature point, the method further includes: The terminal calculates a second calibration error of the projection device; The terminal determines whether the second calibration error is lower than a second preset value; If yes, it is considered as qualified calibration and jumps to the output step; If not, it is considered as calibration failure and jumps to the second acquisition step.

7. The calibration method according to claim 1, characterized in that: After the output step, the method further includes: Judgment step: The terminal establishes a quantitative evaluation standard and judges whether the calibration file is qualified according to the quantitative evaluation standard.

8. The calibration method according to claim 7, characterized in that: The step of judging whether the calibration file is qualified according to the quantitative evaluation standard includes: The terminal obtains the three-dimensional space point corresponding to the feature point; The terminal constructs a triangular mesh according to the three-dimensional space points; The terminal obtains the complete number of the triangular mesh; The terminal determines whether the complete number of the triangular mesh is greater than a third preset value.

9. A calibration system, characterized in that: The calibration system comprises: a terminal, a 3D scanner, and an auxiliary camera, wherein the 3D scanner comprises a scanner camera and a projection device, the frame of the auxiliary camera is larger than the frame of the scanner camera, and the resolution of the auxiliary camera is larger than the resolution of the scanner camera; The scanner camera is used to collect the calibration plate image and the projection image; The auxiliary camera is used to collect the calibration plate image and the projection image; The projection device is used to project the coding pattern onto the whiteboard; The terminal is used to perform the following steps: The first acquisition step: acquiring the calibration plate image captured by the auxiliary camera and the scanner camera; The first extraction step: extracting characteristic corner points of the calibration plate image; Scanner camera calibration step: calculating a first internal parameter and a first external parameter according to the feature corner point, wherein the first internal parameter is an internal parameter of the scanner camera, and the first external parameter is an external parameter of the scanner camera; The second acquisition step: acquiring the projection image captured by the auxiliary camera and the scanner camera, wherein the projection image is an image formed by the projection device projecting onto the whiteboard; The second extraction step: extracting feature points of the projection image; Projection device calibration step: calculating a second internal parameter and a second external parameter according to the first internal parameter, the first external parameter and the feature point, wherein the second internal parameter is an internal parameter of the projection device, and the second external parameter is an external parameter of the projection device; Output step: output the calibration file according to the first internal parameter, the first external parameter, the second internal parameter, and the second external parameter.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the calibration method according to any one of claims 1 to 8.

Images