Binocular camera and scanner calibration method, system, device and storage medium

Abstract

The present invention relates to the field of camera calibration technology in image analysis, and specifically to a calibration method, system, device and storage medium for a binocular camera and a scanner. To achieve accurate calibration of a binocular camera and a scanner, the present invention firstly obtains a plane position set of a reflective point by attaching a reflective sticker on a scanner frame and using an image of the reflective sticker acquired by a camera. Then, based on the difference in horizontal coordinate data of the same reflective point in a first camera and a second camera, the depth position of the reflective point is obtained, thereby forming a spatial position data set of the reflective point. Finally, according to the spatial relationship between the binocular camera and the scanner frame, and the spatial relationship between the scanner and a calibration plate and the scanner frame, a conversion matrix from a camera coordinate system to a scanner coordinate system is obtained, and the binocular camera and the scanner are calibrated. The binocular camera and the scanner are used in an industrial parts scanning scene, and accurate alignment of scanner data and binocular camera data can be achieved, thereby improving the accuracy of subsequent three-dimensional data reconstruction.

Description

Technical Field

[0001] The present invention relates to the technical field of camera calibration in image analysis, and in particular to a calibration method, system, device and storage medium for a binocular camera and a scanner. Background Art

[0002] Laser scanning data can record the geometric properties and position information of the object surface, and the camera's two-dimensional image can record the color information and texture information of the object surface. The combination of the two forms point cloud data, which can more accurately describe the real world. Therefore, the calibration of scanners and cameras is a key technology that determines the accuracy of point cloud data. However, the camera and scanner are two separate systems, and there is no common part between the two sensor data. It is difficult to find the features and geometric constraints that directly correspond between the camera image and the scanner data, resulting in large data errors, poor data alignment, and low data accuracy when the scanner and camera work together. Summary of the invention

[0003] The object of the present invention is to provide a calibration method, system, device and storage medium for a binocular camera and a scanner suitable for use in large-size industrial parts scanning scenarios.

[0004] The technical solution of the present invention is as follows:

[0005] A method for calibrating a binocular camera and a scanner comprises the following operations:

[0006] S1, the first camera and the second camera respectively obtain images of the reflective points outside the scanner frame to obtain respective reflective sticker images; the respective reflective sticker images are subjected to binarization processing, marker point connected domain analysis and circle fitting processing, and the center coordinates of each fitting circle are obtained to obtain the central plane position of the respective reflective points; the central plane positions of all the reflective points of the first camera and the second camera respectively form the reflective point plane position set of the first camera coordinate system and the reflective point plane position set of the second camera coordinate system;

[0007] S2, based on the plane position data of each reflective point in the first camera coordinate system plane position set, the plane position data corresponding to the plane position data of each reflective point in the second camera coordinate system plane position set, obtaining the depth position of each reflective point; the first camera coordinate system reflective point plane position set and the second camera coordinate system reflective point plane position set, respectively, and the corresponding reflective point depth positions, form a first camera coordinate system reflective point spatial position data set and a second camera coordinate system reflective point spatial position data set;

[0008] S3. Based on the spatial position dataset of the reflection point in the first camera coordinate system and the spatial position dataset of the reflection point in the second camera coordinate system, respectively, obtain the transformation matrix from the first camera coordinate system to the scanner frame coordinate system, and the transformation matrix from the second camera coordinate system to the scanner frame coordinate system, and obtain the transformation matrices from the first camera coordinate system and the second camera coordinate system to the scanner coordinate system, respectively, for calibrating the first camera, the second camera and the scanner, respectively.

[0009] The operations for obtaining the center plane position of the reflective point in S1 are as follows: the reflective sticker image is binarized several times to obtain several binary reflective stickers; connected domain analysis and center point fitting processing are performed on the marked points in each binary reflective sticker to obtain several preliminary center points in each binary reflective sticker; based on the position information of each preliminary center point in the corresponding binary reflective sticker, the preliminary center points within the same neighborhood are divided into a center point group; circle fitting processing is performed on each center point group, and the center coordinates of each fitted circle are obtained to obtain the center plane position of the reflective point in the reflective sticker image.

[0010] During the binarization processes, the grayscale threshold of each binarization process is different; the grayscale threshold of the current binarization process is the sum of the grayscale threshold of the previous binarization process and the preset threshold step.

[0011] After performing a connected domain analysis on the marked points in each binary reflective map to obtain a number of white connected domain areas, the operation also includes area filtering on the several white connected domain areas, retaining the white connected domain areas within the preset connected domain area range, and obtaining a number of selected white connected domain areas in each binary reflective map for performing a center point fitting operation.

[0012] In S2, the operation of obtaining the depth position of the current reflecting point is specifically as follows: obtaining the horizontal coordinate data corresponding to the current reflecting point in the first camera coordinate system reflecting point plane position set and the second camera coordinate system reflecting point plane position set, and obtaining the first camera horizontal coordinate and the second camera horizontal coordinate of the current reflecting point; based on the difference between the first camera horizontal coordinate and the second camera horizontal coordinate of the current reflecting point, the distance between the first camera and the second camera, and the focal length of the first camera or the second camera, obtaining the depth position of the current reflecting point.

[0013] The operation of obtaining the conversion matrix from the first camera coordinate system to the scanner coordinate system in S3 is specifically as follows: based on the relationship between the coordinate values ​​of the calibrated points in the calibration plate coordinate system and the coordinate values ​​of the calibrated points in the scanner coordinate system, and the conversion matrix from the scanner coordinate system to the calibration plate coordinate system, the scanner viewing angle relationship is obtained; based on the relationship between the coordinate values ​​of the calibrated points in the calibration plate coordinate system and the coordinate values ​​of the calibrated points in the first camera coordinate system, the conversion matrix from the first camera coordinate system to the scanner frame coordinate system, the conversion matrix from the scanner coordinate system to the scanner frame coordinate system, and the conversion matrix from the scanner coordinate system to the calibration plate coordinate system, the first camera viewing angle relationship is obtained; based on the relationship between the coordinate values ​​of the calibrated points in the scanner coordinate system and the coordinate values ​​of the calibrated points in the first camera coordinate system, and the conversion matrix from the first camera coordinate system to the scanner coordinate system, the camera-scanner relationship is obtained; based on the scanner viewing angle relationship, the first camera viewing angle relationship and the camera-scanner relationship, a set of equations is constructed to solve the conversion matrix from the first camera coordinate system to the scanner coordinate system.

[0014] The scanner viewing angle relationship is as follows:

[0015] ,

[0016] is the coordinate value of the calibration point on the calibration plate coordinate system, is the coordinate value of the fixed point in the first camera coordinate system, is the transformation matrix from the first camera coordinate system to the scanner frame coordinate system;

[0017] The first camera perspective relationship is as follows:

[0018] ,

[0019] is the coordinate value of the fixed point in the first camera coordinate system, is the transformation matrix from the first camera coordinate system to the scanner frame coordinate system, is the inverse of the transformation matrix from the scanner coordinate system to the scanner frame coordinate system;

[0020] The camera scanner relationship is as follows:

[0021] ,

[0022] is the transformation matrix from the first camera coordinate system to the scanner coordinate system.

[0023] A binocular camera and scanner calibration system, used to implement the above binocular camera and scanner calibration method, comprising:

[0024] A reflective point plane position set generation module is used for the first camera and the second camera to respectively obtain images of reflective points outside the scanner frame to obtain respective reflective sticker images; the respective reflective sticker images are subjected to binarization processing, marker point connected domain analysis and circle fitting processing, and the coordinates of the center of each fitting circle are obtained to obtain the respective reflective point center plane positions; all the reflective point center plane positions of the first camera and the second camera respectively form the reflective point plane position set of the first camera coordinate system and the reflective point plane position set of the second camera coordinate system;

[0025] A module for generating a spatial position data set of a reflective point is used to obtain the depth position of each reflective point based on the plane position data corresponding to each plane position data of the reflective point in the plane position set of the second camera coordinate system based on the plane position set of the reflective point in the first camera coordinate system; the plane position set of the reflective point in the first camera coordinate system and the second camera coordinate system reflective point plane position set are respectively combined with the corresponding depth positions of the reflective points to form a spatial position data set of the reflective point in the first camera coordinate system and a spatial position data set of the reflective point in the second camera coordinate system;

[0026] The coordinate system transformation matrix generation and camera and scanner calibration module is used to obtain the transformation matrix from the first camera coordinate system to the scanner frame coordinate system and the transformation matrix from the second camera coordinate system to the scanner frame coordinate system based on the first camera coordinate system reflection point spatial position data set and the second camera coordinate system reflection point spatial position data set, and obtain the transformation matrices from the first camera coordinate system and the second camera coordinate system to the scanner coordinate system, respectively, for calibrating the first camera, the second camera and the scanner, respectively.

[0027] A binocular camera and scanner calibration device comprises a processor and a memory, wherein the processor implements the binocular camera and scanner calibration method when executing a computer program stored in the memory.

[0028] A computer-readable storage medium is used to store a computer program, wherein the computer program implements the above-mentioned calibration method for a binocular camera and a scanner when executed by a processor.

[0029] The beneficial effects of the present invention are:

[0030] The present invention provides a calibration method for a binocular camera and a scanner. First, a reflective tape is attached to a scanner frame, and a camera is used to capture an image of a reflective point outside the scanner frame to obtain a reflective tape image. The reflective tape image is subjected to binarization processing, a connected domain analysis of a marker point, and a circle fitting processing to obtain a plane position set of the reflective points. Then, based on the difference in horizontal coordinate data of the same reflective point in a first camera and a second camera, the depth position of the reflective point is obtained. Combined with the plane position information of the reflective point, the spatial position of each reflective point in the first camera coordinate system and the second camera coordinate system is obtained, thereby forming the spatial position data of the reflective point in the first camera coordinate system. A data set of the spatial position of the reflection points in the first camera coordinate system and the second camera coordinate system is obtained; finally, according to the spatial relationship between the camera and the scanner frame, the transformation matrix between the camera coordinate system and the scanner frame coordinate system is obtained, and based on the transformation matrix from the dual camera coordinate system to the scanner frame coordinate system, the transformation matrix from the scanner coordinate system to the calibration plate coordinate system, and the transformation matrix from the scanner coordinate system to the scanner frame coordinate system, the transformation matrix from the camera coordinate system to the scanner coordinate system is obtained, and the first camera, the second camera and the scanner are calibrated. For use in industrial parts scanning scenarios, the scanner and the binocular camera can be accurately aligned to obtain data, thereby improving the accuracy of subsequent three-dimensional data reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] By reading the detailed description of the preferred embodiment below, the scheme and advantages of the present application will become clear to those skilled in the art. The accompanying drawings are only for the purpose of illustrating the preferred embodiment and are not to be considered as limiting the present invention.

[0032] In the attached picture:

[0033] Figure 1 In the embodiment, a schematic diagram of the flow of the calibration method of this embodiment;

[0034] Figure 2 : is a diagram showing the effect of the reflective sticker on the scanner frame in this embodiment. DETAILED DESCRIPTION

[0035] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings.

[0036] This embodiment is suitable for the calibration of a binocular camera and a scanner in a large-size industrial parts scanning scenario. The left camera and the right camera (i.e., the first camera and the second camera) in the binocular camera are used to obtain the target image, and the scanner (including but not limited to a laser scanner) is used to obtain the target three-dimensional data. The scanner is placed inside the scanner frame, and the reflective sticker is attached to the outside of the scanner frame, which has a similar function to the calibration plate and serves as a reference target to provide calibration reference data.

[0037] This embodiment provides a calibration method for a binocular camera and a scanner, see Figure 1 , including the following operations:

[0038] S1, the first camera and the second camera respectively obtain images of the reflective points outside the scanner frame to obtain respective reflective sticker images; the respective reflective sticker images are subjected to binarization processing, marker point connected domain analysis and circle fitting processing, and the center coordinates of each fitting circle are obtained to obtain the central plane position of the respective reflective points; the central plane positions of all the reflective points of the first camera and the second camera respectively form the reflective point plane position set of the first camera coordinate system and the reflective point plane position set of the second camera coordinate system;

[0039] S2, based on the plane position data of each reflective point in the first camera coordinate system plane position set, the plane position data corresponding to the plane position data of each reflective point in the second camera coordinate system plane position set, obtaining the depth position of each reflective point; the first camera coordinate system reflective point plane position set and the second camera coordinate system reflective point plane position set, respectively, and the corresponding reflective point depth positions, form a first camera coordinate system reflective point spatial position data set and a second camera coordinate system reflective point spatial position data set;

[0040] S3. Based on the spatial position dataset of the reflection point in the first camera coordinate system and the spatial position dataset of the reflection point in the second camera coordinate system, respectively, obtain the transformation matrix from the first camera coordinate system to the scanner frame coordinate system, and the transformation matrix from the second camera coordinate system to the scanner frame coordinate system, and obtain the transformation matrices from the first camera coordinate system and the second camera coordinate system to the scanner coordinate system, respectively, for calibrating the first camera, the second camera and the scanner, respectively.

[0041] S1, the first camera and the second camera respectively obtain images of the reflective points outside the scanner frame to obtain their own reflective sticker images; each reflective sticker image is processed by binarization, connected domain analysis of marked points and circle fitting, and the center coordinates of each fitting circle are obtained to obtain the center plane position of each reflective point; all the center plane positions of the reflective points of the first camera and the second camera form the reflective point plane position set of the first camera coordinate system and the reflective point plane position set of the second camera coordinate system, respectively.

[0042] By sticking reflective tape on the scanner frame, the camera (the first camera or the second camera) captures the image of the reflective points outside the scanner frame to obtain the reflective tape image, and the reflective tape image is binarized, the marker point connected domain analysis and circle fitting processing are performed to obtain the plane position set of the reflective points, which is used to subsequently obtain the spatial position relationship between the camera and the scanner frame.

[0043] The method for obtaining the above-mentioned reflective point plane position set is specifically as follows. Since the methods for obtaining the reflective point plane position set of the first camera and the second camera are similar, in order to save space, the first camera or the second camera is replaced by camera below.

[0044] First, a reflective tape is attached to the scanner frame (the scanner is located inside the scanner frame). The reflective tape contains several reflective points. The camera captures the image of the reflective points outside the scanner frame and obtains the reflective tape image, which is used to analyze the positional relationship between the camera and the scanner frame. The reflective points in the reflective tape are circular reflective points and are fixed to the scanner frame (see Figure 2 The function of the reflective sticker is to reflect the infrared light emitted by the infrared light source back to the camera so that the camera can better capture the reflective marking points.

[0045] Then, the reflective sticker image is binarized, and the position points with grayscale values ​​greater than the grayscale threshold (usually reflective points with strong reflectivity and high brightness) are marked as white to obtain several marked points; several marked points are analyzed by the connected domain of the marked points to form several white connected domain areas; several white connected domain areas are respectively processed by circle fitting, and the center coordinates of each fitting circle are obtained to obtain the two-dimensional information of the reflective point in the camera coordinate system (position information in the x and y coordinate directions), and the central plane position of the reflective point is obtained.

[0046] Among them, in order to further improve the accuracy of the center plane position of the reflective point, the above operation of obtaining the center plane position of the reflective point is specifically as follows: the reflective sticker image is binarized several times respectively, and the white point mark image of the reflective sticker image under different grayscale thresholds is obtained to obtain several binary reflective maps, and each binary reflective map contains several white mark points; for the mark points in each binary reflective map, connected domain analysis and center point fitting processing are performed respectively, that is, connected domain analysis is performed on the mark points in each binary reflective map, and the white mark points in the same area of ​​interest are divided into a white connected domain area, and A circle fitting process is performed on each white connected domain area to obtain the center of the fitted circle as the center point of the white connected domain area, and each binary reflective map contains several preliminary center points; based on the position information of each preliminary center point in the corresponding binary reflective map, the preliminary center points whose positions are within the same neighborhood are divided into a center point group, and the corresponding position data form a center point group position set; according to the center point group position set of each center point group, a circle fitting process is performed on each center point group, and the center coordinates of each fitted circle are obtained to improve the accuracy of the center data and obtain the center plane position of the reflective point in the reflective map image.

[0047] Among them, during several binarization processes, the grayscale threshold of each binarization process is different; the grayscale threshold of the current binarization process is the sum of the grayscale threshold of the previous binarization process and the preset threshold step. The grayscale threshold of the first binarization process is the preset minimum grayscale threshold, and the grayscale threshold of the last binarization process is the preset maximum grayscale threshold. That is: set a minimum grayscale threshold, a maximum grayscale threshold, and a threshold step, and take a series of grayscale thresholds from the minimum grayscale threshold to the maximum grayscale threshold according to the threshold step, which is used to perform several binarization processes with different grayscale thresholds on the reflective sticker image, and obtain several binary reflective stickers, so as to obtain the characteristics of the reflective points in the reflective sticker image during different binarization processes.

[0048] Furthermore, in order to reduce the impact of data noise and further improve the accuracy of the central plane position of the reflective point, a connected domain analysis is performed on the marked points in each binary reflective map to obtain a number of white connected domain areas, which also includes area filtering of the several white connected domain areas, retaining the white connected domain areas whose areas are within the preset connected domain area range, and obtaining each binary reflective map contains a number of selected white connected domain areas for performing the center point fitting operation.

[0049] Finally, the center plane positions of all reflective points form a reflective point plane position set, which is used to subsequently obtain the positional relationship between the camera and the scanner frame, making it easier to calibrate the camera.

[0050] S2. Based on the plane position set of reflective points in the first camera coordinate system, the plane position data of each reflective point corresponds to the plane position data in the plane position set of reflective points in the second camera coordinate system, and the depth position of each reflective point is obtained; the plane position set of reflective points in the first camera coordinate system and the plane position set of reflective points in the second camera coordinate system, respectively, together with the corresponding depth positions of the reflective points, form a first camera coordinate system reflective point spatial position data set and a second camera coordinate system reflective point spatial position data set.

[0051] Based on the difference in the horizontal coordinate data of the same reflective point in the first camera and the second camera, the depth position of the reflective point is obtained, and combined with the planar position information of the reflective point, the spatial position of each reflective point in the first camera coordinate system and the second camera coordinate system is obtained, forming a spatial position data set of the reflective point in the first camera coordinate system and a spatial position data set of the reflective point in the second camera coordinate system, which is convenient for subsequently obtaining the spatial position relationship between the camera and the scanner frame.

[0052] Based on the plane position set of the reflection points in the first camera coordinate system, the plane position data of each reflection point corresponds to the plane position data in the plane position set of the reflection points in the second camera coordinate system, and the depth position (position information in the z-axis direction) of each reflection point is obtained.

[0053] The operation of obtaining the depth position of the current reflection point is specifically as follows: obtaining the horizontal coordinate data corresponding to the current reflection point in the reflection point plane position set of the first camera coordinate system and the reflection point plane position set of the second camera coordinate system, and obtaining the first camera horizontal coordinate and the second camera horizontal coordinate of the current reflection point; based on the difference between the first camera horizontal coordinate and the second camera horizontal coordinate of the current reflection point, the distance between the first camera and the second camera, and the focal length of the first camera or the second camera, obtaining the depth position of the current reflection point.

[0054] Among them, when the center of the first camera and the center of the second camera after stereo correction are at the same horizontal height, and the focal lengths of the first camera and the second camera are equal, the depth position of the current reflection point satisfies the following relationship:

[0055] ,

[0056] is the distance between the first camera and the second camera, is the depth position of the current reflection point, , They are the horizontal coordinates of the first camera and the second camera of the current reflection point, is the focal length of the first camera or the second camera.

[0057] Based on the above relationship, the depth position formula of the current reflection point can be obtained as follows:

[0058] .

[0059] The plane position set of the reflecting points in the first camera coordinate system (the plane position data of the reflecting points in the first camera coordinate system - data in the x-axis and y-axis directions) and the plane position set of the reflecting points in the second camera coordinate system (the plane position data of the reflecting points in the second camera coordinate system - data in the x-axis and y-axis directions), respectively, together with the corresponding depth positions of the reflecting points (the data of the reflecting points in the z-axis direction in the camera coordinate system), form the first camera coordinate system reflecting point spatial position data set and the second camera coordinate system reflecting point spatial position data set, which are used to subsequently obtain the spatial position relationship between the camera and the scanner frame.

[0060] S3. Based on the spatial position dataset of the reflection point in the first camera coordinate system and the spatial position dataset of the reflection point in the second camera coordinate system, respectively, obtain the transformation matrix from the first camera coordinate system to the scanner frame coordinate system, and the transformation matrix from the second camera coordinate system to the scanner frame coordinate system, and obtain the transformation matrices from the first camera coordinate system and the second camera coordinate system to the scanner coordinate system, respectively, for calibrating the first camera, the second camera and the scanner, respectively.

[0061] According to the spatial relationship between the binocular camera and the scanner frame, the transformation matrix between the binocular camera coordinate system and the scanner frame coordinate system is obtained; based on the transformation matrix from the binocular camera coordinate system to the scanner frame coordinate system, the transformation matrix from the scanner coordinate system to the calibration plate coordinate system, and the transformation matrix from the scanner coordinate system to the scanner frame coordinate system, the transformation matrix from the camera coordinate system to the scanner coordinate system is obtained, and the first camera, the second camera and the scanner are calibrated.

[0062] Firstly, based on the spatial position dataset of the reflection point in the first camera coordinate system and the spatial position dataset of the reflection point in the second camera coordinate system, a transformation matrix from the first camera coordinate system to the scanner frame coordinate system and a transformation matrix from the second camera coordinate system to the scanner frame coordinate system are obtained respectively.

[0063] Then, based on the transformation matrix from the first camera coordinate system to the scanner frame coordinate system, the transformation matrix from the second camera coordinate system to the scanner frame coordinate system, the transformation matrix from the scanner coordinate system to the calibration plate coordinate system, and the transformation matrix from the scanner coordinate system to the scanner frame coordinate system, the transformation matrix from the first camera coordinate system to the scanner coordinate system and the transformation matrix from the second camera coordinate system to the scanner coordinate system are obtained respectively.

[0064] The operation of obtaining the transformation matrix from the first camera coordinate system to the scanner coordinate system is as follows.

[0065] Step 1: Based on the relationship between the coordinate values ​​of the calibration points in the calibration plate coordinate system and the coordinate values ​​of the calibration points in the scanner coordinate system, and the transformation matrix from the scanner coordinate system to the calibration plate coordinate system, the scanner viewing angle relationship is obtained.

[0066] For a calibration point on the calibration plate, the coordinates of the calibration point in the calibration plate coordinate system are In the scanner coordinate system, after the coordinate system transformation, the coordinate value of the calibration point in the calibration plate coordinate system is , and The relationship between the scanner viewing angle is as follows:

[0067] ,

[0068] is the coordinate value of the calibration point on the calibration plate coordinate system, is the coordinate value of the fixed point in the first camera coordinate system, is the transformation matrix from the first camera coordinate system to the scanner frame coordinate system.

[0069] Step 2: Based on the relationship between the coordinate values ​​of the calibration points on the calibration plate coordinate system and the coordinate values ​​of the calibration points in the first camera coordinate system, the transformation matrix from the first camera coordinate system to the scanner frame coordinate system, the transformation matrix from the scanner coordinate system to the scanner frame coordinate system, and the transformation matrix from the scanner coordinate system to the calibration plate coordinate system, the first camera viewing angle relationship is obtained.

[0070] From the perspective of the first camera, the coordinates of the calibration point on the calibration plate in the first camera coordinate system are The scanner frame coordinate system satisfies the following equation:

[0071] ,

[0072] is the coordinate value of the fixed point in the first camera coordinate system, is the transformation matrix from the first camera coordinate system to the scanner frame coordinate system, is the inverse of the transformation matrix from the scanner coordinate system to the scanner frame coordinate system.

[0073] Substituting the scanner viewing angle relationship into the above formula, the first camera viewing angle relationship is as follows: .

[0074] Step 3: Based on the relationship between the coordinate values ​​of the calibrated points in the scanner coordinate system and the coordinate values ​​of the calibrated points in the first camera coordinate system, and the transformation matrix from the first camera coordinate system to the scanner coordinate system, a camera-scanner relationship is obtained.

[0075] That is, when the first camera and the scanner are calibrated to reach an ideal state, the first camera and the scanner satisfy the following camera-scanner relationship: , is the relationship of the transformation matrix from the first camera coordinate system to the scanner coordinate system.

[0076] Step 4: Based on the scanner perspective relationship, the first camera perspective relationship and the camera scanner relationship, construct a set of equations and solve them to obtain the transformation matrix from the first camera coordinate system to the scanner coordinate system. Substitute the scanner perspective relationship and the first camera perspective relationship into the camera scanner relationship to obtain , the transformation matrix from the first camera coordinate system to the scanner frame coordinate system , the transformation matrix from the scanner coordinate system to the calibration plate coordinate system , the transformation matrix from the scanner coordinate system to the scanner frame coordinate system , Substitute into the above formula to get the transformation matrix from the first camera coordinate system to the scanner coordinate system .

[0077] The method for obtaining the transformation matrix from the second camera coordinate system to the scanner coordinate system is the same as the method for obtaining the transformation matrix from the first camera coordinate system to the scanner coordinate system, and will not be repeated here to save space.

[0078] Finally, based on the transformation matrix from the first camera coordinate system to the scanner coordinate system and the transformation matrix from the second camera coordinate system to the scanner coordinate system, the first camera, the second camera and the scanner are calibrated respectively. For use in industrial parts scanning scenarios, the scanner and the binocular camera can be accurately aligned to obtain data, thereby improving the accuracy of subsequent three-dimensional data reconstruction.

[0079] This embodiment further provides a binocular camera and scanner calibration system, which is used to implement the above binocular camera and scanner calibration method, including:

[0080] A reflective point plane position set generation module is used for the first camera and the second camera to respectively obtain images of reflective points outside the scanner frame to obtain respective reflective sticker images; the respective reflective sticker images are subjected to binarization processing, marker point connected domain analysis and circle fitting processing, and the coordinates of the center of each fitting circle are obtained to obtain the respective reflective point center plane positions; all the reflective point center plane positions of the first camera and the second camera respectively form the reflective point plane position set of the first camera coordinate system and the reflective point plane position set of the second camera coordinate system;

[0081] A module for generating a spatial position data set of a reflective point is used to obtain the depth position of each reflective point based on the plane position data corresponding to each plane position data of the reflective point in the plane position set of the second camera coordinate system based on the plane position set of the reflective point in the first camera coordinate system; the plane position set of the reflective point in the first camera coordinate system and the second camera coordinate system reflective point plane position set are respectively combined with the corresponding depth positions of the reflective points to form a spatial position data set of the reflective point in the first camera coordinate system and a spatial position data set of the reflective point in the second camera coordinate system;

[0082] The coordinate system conversion matrix generation and camera and scanner calibration module is used to obtain the conversion matrix from the first camera coordinate system to the scanner frame coordinate system and the conversion matrix from the second camera coordinate system to the scanner frame coordinate system based on the first camera coordinate system reflection point spatial position data set and the second camera coordinate system reflection point spatial position data set respectively; based on the conversion matrix from the first camera coordinate system to the scanner frame coordinate system, the conversion matrix from the second camera coordinate system to the scanner frame coordinate system, the conversion matrix from the scanner coordinate system to the calibration plate coordinate system, and the conversion matrix from the scanner coordinate system to the scanner frame coordinate system, respectively obtain the conversion matrix from the first camera coordinate system to the scanner coordinate system and the conversion matrix from the second camera coordinate system to the scanner coordinate system; based on the conversion matrix from the first camera coordinate system to the scanner coordinate system and the conversion matrix from the second camera coordinate system to the scanner coordinate system, respectively calibrate the first camera, the second camera and the scanner.

[0083] This embodiment also provides a binocular camera and scanner calibration device, including a processor and a memory, wherein the processor implements the above-mentioned binocular camera and scanner calibration method when executing a computer program stored in the memory.

[0084] This embodiment also provides a computer-readable storage medium for storing a computer program, wherein the computer program implements the above-mentioned calibration method of the binocular camera and the scanner when executed by a processor.

[0085] The present embodiment provides a calibration method for a binocular camera and a scanner. First, a reflective tape is attached to a scanner frame, and a camera is used to capture an image of a reflective point outside the scanner frame to obtain a reflective tape image. The reflective tape image is subjected to binarization processing, connected domain analysis of marker points, and circle fitting processing to obtain a plane position set of the reflective points. Then, based on the difference in horizontal coordinate data of the same reflective point in the first camera and the second camera, the depth position of the reflective point is obtained. Combined with the plane position information of the reflective point, the spatial position of each reflective point in the first camera coordinate system and the second camera coordinate system is obtained, forming a spatial position data set of the reflective point in the first camera coordinate system. The data set is composed of the first camera, the second camera and the spatial position data set of the reflection points in the second camera coordinate system; finally, according to the spatial relationship between the camera and the scanner frame, the transformation matrix between the camera coordinate system and the scanner frame coordinate system is obtained, and based on the transformation matrix from the dual camera coordinate system to the scanner frame coordinate system, the transformation matrix from the scanner coordinate system to the calibration plate coordinate system, and the transformation matrix from the scanner coordinate system to the scanner frame coordinate system, the transformation matrix from the camera coordinate system to the scanner coordinate system is obtained, and the first camera, the second camera and the scanner are calibrated. For use in industrial parts scanning scenarios, the scanner and the binocular camera can be accurately aligned to obtain data, thereby improving the accuracy of subsequent three-dimensional data reconstruction.

Claims

1. A calibration method for a binocular camera and a scanner, characterized in that: The following operations are included: S1, the first camera and the second camera respectively obtain images of the reflective points outside the scanner frame to obtain respective reflective sticker images; the respective reflective sticker images are subjected to binarization processing, marker point connected domain analysis and circle fitting processing, and the center coordinates of each fitting circle are obtained to obtain the central plane position of the respective reflective points; the central plane positions of all the reflective points of the first camera and the second camera respectively form the reflective point plane position set of the first camera coordinate system and the reflective point plane position set of the second camera coordinate system; S2, based on the plane position data of each reflective point in the first camera coordinate system plane position set, the plane position data corresponding to the plane position data of each reflective point in the second camera coordinate system plane position set, obtaining the depth position of each reflective point; the first camera coordinate system reflective point plane position set and the second camera coordinate system reflective point plane position set, respectively, and the corresponding reflective point depth positions, form a first camera coordinate system reflective point spatial position data set and a second camera coordinate system reflective point spatial position data set; S3. Based on the spatial position dataset of the reflection point in the first camera coordinate system and the spatial position dataset of the reflection point in the second camera coordinate system, respectively, obtain the transformation matrix from the first camera coordinate system to the scanner frame coordinate system, and the transformation matrix from the second camera coordinate system to the scanner frame coordinate system, and obtain the transformation matrices from the first camera coordinate system and the second camera coordinate system to the scanner coordinate system, respectively, for calibrating the first camera, the second camera and the scanner, respectively.

2. The calibration method of binocular camera and scanner according to claim 1, characterized in that: In S1, the operation of obtaining the center plane position of the reflection point is specifically as follows: The reflective sticker image is subjected to several binarization processes to obtain several binary reflective sticker images; Conducting connected domain analysis and center point fitting processing on the marked points in each binary reflective map, and obtaining a number of preliminary center points in each binary reflective map; Based on the position information of each preliminary center point in the corresponding binary reflective map, the preliminary center points whose positions are within the same neighborhood are divided into a center point group; Perform circle fitting processing on each center point group, obtain the coordinates of the center of each fitting circle, and obtain the center plane position of the reflective point in the reflective sticker image.

3. The calibration method of binocular camera and scanner according to claim 2, characterized in that: During the binarization processes, the grayscale threshold of each binarization process is different; the grayscale threshold of the current binarization process is the sum of the grayscale threshold of the previous binarization process and the preset threshold step.

4. The calibration method of binocular camera and scanner according to claim 2, characterized in that: After performing a connected domain analysis on the marked points in each binary reflective map to obtain a number of white connected domain areas, the operation also includes area filtering on the several white connected domain areas, retaining the white connected domain areas within the preset connected domain area range, and obtaining a number of selected white connected domain areas in each binary reflective map for performing a center point fitting operation.

5. The calibration method of binocular camera and scanner according to claim 1, characterized in that: In S2, the operation of obtaining the depth position of the current reflection point is specifically as follows: Obtain the horizontal coordinate data corresponding to the current reflection point in the reflection point plane position set of the first camera coordinate system and the reflection point plane position set of the second camera coordinate system, and obtain the first camera horizontal coordinate and the second camera horizontal coordinate of the current reflection point; The depth position of the current reflection point is obtained based on the difference between the horizontal coordinates of the first camera and the horizontal coordinates of the second camera of the current reflection point, the distance between the first camera and the second camera, and the focal length of the first camera or the second camera.

6. The calibration method of binocular camera and scanner according to claim 1, characterized in that: In S3, the operation of obtaining the transformation matrix from the first camera coordinate system to the scanner coordinate system is specifically: Based on the relationship between the coordinate values ​​of the calibration points in the calibration plate coordinate system and the coordinate values ​​of the calibration points in the scanner coordinate system, and the conversion matrix from the scanner coordinate system to the calibration plate coordinate system, a scanner viewing angle relationship is obtained; Based on the relationship between the coordinate values ​​of the calibration points on the calibration plate coordinate system and the coordinate values ​​of the calibration points in the first camera coordinate system, the transformation matrix from the first camera coordinate system to the scanner frame coordinate system, the transformation matrix from the scanner coordinate system to the scanner frame coordinate system, and the transformation matrix from the scanner coordinate system to the calibration plate coordinate system, a first camera viewing angle relationship is obtained; Based on the relationship between the coordinate value of the calibrated point in the scanner coordinate system and the coordinate value of the calibrated point in the first camera coordinate system, and the transformation matrix from the first camera coordinate system to the scanner coordinate system, a camera-scanner relationship is obtained; Based on the scanner viewing angle relationship, the first camera viewing angle relationship and the camera scanner relationship, a set of equations is constructed and the transformation matrix from the first camera coordinate system to the scanner coordinate system is obtained by solving the equations.

7. The calibration method of binocular camera and scanner according to claim 6, characterized in that: The scanner viewing angle relationship is as follows: ， is the coordinate value of the calibration point on the calibration plate coordinate system, is the coordinate value of the fixed point in the first camera coordinate system, is the transformation matrix from the first camera coordinate system to the scanner frame coordinate system; The first camera perspective relationship is as follows: ， is the coordinate value of the fixed point in the first camera coordinate system, is the transformation matrix from the first camera coordinate system to the scanner frame coordinate system, is the inverse of the transformation matrix from the scanner coordinate system to the scanner frame coordinate system; The camera scanner relationship is as follows: ， is the transformation matrix from the first camera coordinate system to the scanner coordinate system.

8. A binocular camera and scanner calibration system, used to implement the binocular camera and scanner calibration method according to claim 1, characterized in that: include: A reflective point plane position set generation module is used for the first camera and the second camera to respectively obtain images of reflective points outside the scanner frame to obtain respective reflective sticker images; The respective reflective sticker images are subjected to binarization processing, marker point connected domain analysis and circle fitting processing, and the coordinates of the center of each fitted circle are obtained to obtain the central plane position of each reflective point; the central plane positions of all the reflective points of the first camera and the second camera respectively form a reflective point plane position set of the first camera coordinate system and a reflective point plane position set of the second camera coordinate system; A module for generating a spatial position data set of a reflective point is used to obtain the depth position of each reflective point based on the plane position data corresponding to each plane position data of the reflective point in the plane position set of the second camera coordinate system based on the plane position set of the reflective point in the first camera coordinate system; the plane position set of the reflective point in the first camera coordinate system and the second camera coordinate system reflective point plane position set are respectively combined with the corresponding depth positions of the reflective points to form a spatial position data set of the reflective point in the first camera coordinate system and a spatial position data set of the reflective point in the second camera coordinate system; The coordinate system transformation matrix generation and camera and scanner calibration module is used to obtain the transformation matrix from the first camera coordinate system to the scanner frame coordinate system and the transformation matrix from the second camera coordinate system to the scanner frame coordinate system based on the first camera coordinate system reflection point spatial position data set and the second camera coordinate system reflection point spatial position data set, and obtain the transformation matrices from the first camera coordinate system and the second camera coordinate system to the scanner coordinate system, respectively, for calibrating the first camera, the second camera and the scanner, respectively.

9. A calibration device for a binocular camera and a scanner, characterized in that: The method comprises a processor and a memory, wherein when the processor executes the computer program stored in the memory, the calibration method of the binocular camera and the scanner as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein when the computer program is executed by a processor, the calibration method of a binocular camera and a scanner as described in any one of claims 1 to 7 is implemented.

Images