Binocular Camera Calibration Control Method, Device, Equipment and Storage Medium

Abstract

An embodiment of the present invention discloses a binocular camera calibration control method, device, equipment and storage medium. The binocular camera is arranged on a robotic arm for calibration. The method includes: acquiring image data of a checkerboard image captured by the binocular camera, and determining whether the checkerboard captured by the binocular camera is complete according to the image data; if the checkerboard in the image data is incomplete, adjusting the robotic arm so that the binocular camera can capture a complete checkerboard image; judging whether the shooting direction of the binocular camera conforms to a preset direction according to the image data; if the shooting direction of the checkerboard does not conform to the preset direction, adjusting the robotic arm until the shooting direction of the binocular camera conforms to the preset direction for shooting. It realizes the continuous correction of the robotic arm error by the calibration module in the automatic calibration operation, reduces the cumulative error of the long-term operation of the robotic arm, and ensures the accuracy and success rate of binocular calibration.

Description

Technical Field

[0001] The present invention relates to the field of camera calibration, and in particular, to a binocular camera calibration control method, device, equipment, and storage medium. Background Art

[0002] In current binocular camera calibration technologies, most are Zhang Zhengyou checkerboard calibration methods, where a binocular camera is used to photograph a checkerboard. In this calibration method, either the checkerboard or the binocular camera needs to be fixed, and the other moves. By moving one of them, it is ensured that the binocular camera can capture checkerboard images at different angles, thereby calibrating the internal and external parameters of the binocular camera. The success rate of manual calibration is high, but the efficiency is very low. In large-scale production, a robotic arm is generally used to calibrate the binocular camera, that is, the checkerboard or the binocular camera is fixed on the robotic arm, and by setting the program of the robotic arm, the robotic arm can carry the binocular camera or the checkerboard and move to different positions and angles, facilitating the binocular camera to capture checkerboard images at different angles. In the robotic arm calibration method, since the robotic arm can operate continuously without interruption, the efficiency is relatively high. Using a robotic arm in large-scale production can stably improve production efficiency. However, the robotic arm also has its problems. The robotic arm needs to set the knob and movement program, but over time, the movement error of the robotic arm will become larger and larger, resulting in the possibility that the checkerboard images captured by the binocular camera may not meet the requirements for calibrating the camera, such as the angle not meeting the requirements, or the captured checkerboard being incomplete, etc., leading to problems such as calibration failure or poor calibration result accuracy. Summary of the Invention

[0003] In view of this, the present application provides a binocular camera calibration control method. The binocular camera is installed on a robotic arm for calibration. The method includes:

[0004] Obtain the image data of the checkerboard image captured by the binocular camera, and determine whether the checkerboard captured by the binocular camera is complete according to the image data;

[0005] If the checkerboard in the image data is incomplete, adjust the robotic arm so that the binocular camera can capture a complete checkerboard image;

[0006] Judge whether the shooting direction of the binocular camera conforms to a preset direction according to the image data;

[0007] If the shooting direction of the checkerboard does not conform to the preset direction, adjust the robotic arm until the shooting direction of the binocular camera conforms to the preset direction and then perform shooting.

[0008] Further, it further includes:

[0009] When taking pictures, every time a picture is taken in one of the preset directions, that direction is marked as completed, and then move to the next preset direction for taking pictures. When all preset directions are marked as completed, the picture taking ends.

[0010] Furthermore, the image data of the checkerboard images captured by the binocular camera includes the left image captured by the left camera of the binocular camera and the right image captured by the right camera of the binocular camera.

[0011] Judging whether the shooting direction of the binocular camera conforms to the preset direction according to the image data includes:

[0012] Extract the coordinates of the four vertices of the checkerboard in the left image and the right image respectively.

[0013] Calculate the angles corresponding to each vertex in the left image and the right image respectively.

[0014] Select the qualified angles, and judge the matching relationship between the pose of the camera and the preset direction according to whether the selected angles are within the preset angle intervals. Among them, different preset angle intervals correspond one-to-one to different preset directions.

[0015] Furthermore, adjusting the robotic arm until the shooting direction of the binocular camera conforms to the preset direction includes:

[0016] Determine the relative position relationship between the pose of the camera and the preset direction according to the size relationship between the selected angles and the preset angle intervals.

[0017] Control the robotic arm to perform corresponding left-right translation, up-down translation or rotation according to the relative position relationship, so that the camera pose conforms to the preset direction.

[0018] Furthermore, selecting the qualified angles includes:

[0019] Compare the differences between the angles formed by the same vertex in the left image and the right image and a right angle respectively, and select the angle formed by the vertex with a larger difference as the qualified angle.

[0020] Furthermore, the preset direction is any one of directly in front of the checkerboard, directly to the left of the checkerboard, directly to the right of the checkerboard, directly above the checkerboard, directly below the checkerboard, upper left, upper right, lower left and lower right.

[0021] Furthermore, if the checkerboard in the image data is incomplete, adjusting the robotic arm includes:

[0022] Calculate whether the area of the checkerboard in the image data matches the checkerboard. If not, the checkerboard is incomplete, and then control the robotic arm to move the binocular camera until the checkerboard in the image data is complete.

[0023] The embodiment of the present application also provides a binocular camera calibration control device, including:

[0024] A first judgment module, configured to obtain the image data of the checkerboard image captured by the binocular camera, and determine whether the checkerboard captured by the binocular camera is complete according to the image data;

[0025] A first adjustment module, configured to, if the checkerboard in the image data is incomplete, adjust the robotic arm so that the binocular camera can capture a complete checkerboard image;

[0026] A second judgment module, configured to judge whether the shooting direction of the binocular camera conforms to a preset direction according to the image data;

[0027] A second adjustment module, configured to, if the shooting direction of the checkerboard does not conform to the preset direction, adjust the robotic arm until the shooting direction of the binocular camera conforms to the preset direction for shooting.

[0028] The embodiment of the present application also provides a control device, including a processor and a memory. The memory stores a computer program, and when the computer program runs on the processor, it executes the binocular camera calibration control method described above.

[0029] The embodiment of the present application also provides a readable storage medium, which stores a computer program, and when the computer program runs on a processor, it executes the binocular camera calibration control method described above.

[0030] The embodiment of the present invention discloses a binocular camera calibration control method, device, equipment and storage medium. The binocular camera is arranged on the robotic arm for calibration. The method includes: obtaining the image data of the checkerboard image captured by the binocular camera, and determining whether the checkerboard captured by the binocular camera is complete according to the image data; if the checkerboard in the image data is incomplete, adjust the robotic arm so that the binocular camera can capture a complete checkerboard image; judge whether the shooting direction of the binocular camera conforms to a preset direction according to the image data; if the shooting direction of the checkerboard does not conform to the preset direction, adjust the robotic arm until the shooting direction of the binocular camera conforms to the preset direction for shooting. It realizes the continuous correction of the robotic arm error by the calibration module in the automatic calibration operation, reduces the cumulative error of the long-term operation of the robotic arm, and ensures the accuracy and success rate of the binocular calibration. Description of the Drawings

[0031] To more clearly illustrate the technical solution of the present invention, the accompanying drawings required in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as limiting the protection scope of the present invention. In each drawing, similar components are numbered similarly.

[0032] Figure 1 Shows a schematic flow diagram of a binocular camera calibration control method of the present application;

[0033] Figure 2 Shows a schematic diagram of a defective checkerboard image;

[0034] Figure 3 Shows a schematic diagram of corner point extraction of a checkerboard image;

[0035] Figure 4 Shows a schematic diagram of judging the shooting direction of a binocular camera;

[0036] Figure 5 Shows a schematic diagram of the structure of a camera robotic arm calibration control device. Detailed implementation manners

[0037] The technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments.

[0038] The components of the embodiments of the present invention generally described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts fall within the protection scope of the present invention.

[0039] In the following text, the terms "including", "having" and their cognates that may be used in various embodiments of the present invention are only intended to indicate specific features, numbers, steps, operations, elements, components or combinations of the foregoing items, and should not be construed as first excluding the existence of one or more other features, numbers, steps, operations, elements, components or combinations of the foregoing items or increasing the possibility of one or more features, numbers, steps, operations, elements, components or combinations of the foregoing items.

[0040] In addition, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0041] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which various embodiments of the present invention pertain. The terms (such as those defined in a general use dictionary) will be interpreted as having the same meaning as their contextual meaning in the relevant technical field and will not be interpreted as having an idealized meaning or an overly formal meaning unless clearly defined in various embodiments of the present invention.

[0042] The technical solution of this application is applied to an application scenario where a robotic arm holds a binocular camera for calibration. In this application scenario, the robotic arm will, according to a predefined program, send the binocular camera to a corresponding position for shooting. For example, it is necessary to control the binocular camera to shoot a checkerboard in 9 different directions and then perform calibration, and the calibration work is carried out with the shooting and calibration in 9 directions as a cycle. However, after working for a long time, the operation of the robotic arm will inevitably deviate, and workers will be required to perform calibration.

[0043] This application determines whether the shooting direction conforms to the preset specified direction by the images captured by the binocular camera. If not, corresponding methods are used to control the robotic arm to adjust the shooting direction of the binocular camera, so as to achieve the purpose of self-feedback and self-correction.

[0044] Next, specific embodiments are used to explain the technical solution of this application.

[0045] Embodiment 1

[0046] As Figure 1 shown, it is a schematic flowchart of a binocular camera calibration control method of this application. The method includes the following steps:

[0047] Step S100, obtain the image data of the checkerboard image captured by the binocular camera, and determine whether the checkerboard captured by the binocular camera is complete according to the image data.

[0048] When performing binocular camera calibration, calibration can be carried out by shooting a checkerboard image. For this purpose, a complete checkerboard image needs to be captured. Before the binocular camera shoots, it will continuously feedback the image data stream of the current scene in the camera. The image data stream is a frame of images. According to these images, it can be judged whether the captured checkerboard is complete.

[0049] The data of the chessboard captured is known. For example, a 7*6 chessboard means that it has 7 columns and 6 rows, a total of 42 black and white grids forming a rectangular chessboard. The chessboard will have 8*7 grids, a total of 56 corner points. If the number of corner points identified in the captured chessboard image is not 56, it can be determined that the currently captured chessboard image is incomplete. On the contrary, if the number of grids in the captured chessboard image matches the known data, it can be determined that the chessboard image captured by the current binocular camera is complete.

[0050] Step S200: If the chessboard in the image data is incomplete, adjust the robotic arm so that the binocular camera can capture a complete chessboard image.

[0051] When it is determined that the chessboard photographed by the binocular camera is incomplete, the robotic arm needs to be adjusted by translating left and right, up and down, or rotating so that the binocular camera can capture the complete chessboard.

[0052] When the chessboard is incomplete, there will be chessboard image elements at the edge of the entire image, such as Figure 2 As shown in the figure, still taking the 7*6 black and white chessboard as an example, it is obvious that the chessboard in the current captured image is missing 2 columns, and the number of corner points extracted from the image is only 42, and the chessboard image is incomplete. Because the shooting angle is to the left, part of the right side of the chessboard is not captured, so there will be chessboard image elements on the right edge of the entire captured image, indicating that the chessboard is incomplete from the right side. Therefore, when it is determined that the captured chessboard is incomplete, the four edges of the entire captured image are judged to see which edges have chessboard image elements, and the next adjustment direction of the robot arm can be determined.

[0053] For example, if a chessboard image element is found on the right edge, the robot arm needs to translate to the right; if a chessboard image element is found on the left edge, it needs to translate to the left.

[0054] Furthermore, because the above-mentioned problem is caused by the error of the robot arm after working for a long time, only fine-tuning is required for this type of adjustment. Therefore, the distance of each translation is a set fixed fine-tuning step. Every time a step is adjusted, a judgment is made until the chessboard photographed by the binocular camera is complete, then the adjustment of the integrity of the chessboard can be stopped.

[0055] Step S300: determining whether the shooting direction of the binocular camera meets a preset direction according to the image data.

[0056] During camera calibration, the camera is made to photograph the same chessboard in multiple preset directions, and then calibration is performed based on the images captured in these different shooting directions to obtain the internal and external parameters of the binocular camera. The preset directions in this embodiment are illustrated by nine directions: directly in front of, directly to the left of, directly to the right of, directly above, directly below, top left, top right, bottom left, and bottom right of the chessboard.

[0057] Among them, the image data of the chessboard images captured by the binocular camera includes the left image captured by the left camera of the binocular camera and the right image captured by the right camera of the binocular camera. Therefore, for the binocular camera at the same position, two images of the chessboard will be captured.

[0058] Specifically, during actual shooting, the left and right images are obtained by alternately covering one camera of the binocular camera. For example, covering the right camera to capture the left image and covering the left camera to capture the right image.

[0059] At the same time, the shooting positions of the left and right images should be on the same horizontal line. Therefore, it is necessary to ensure the stability of the camera. To ensure the stability of the camera during calibration, the binocular camera can be set on a camera support, and the support is grasped by a robotic arm to perform the shooting calibration operation.

[0060] Specifically, as Figure 4 shown, this step further includes the following sub-steps:

[0061] Step S310, respectively extract the coordinates of the four vertices of the chessboard in the left image and the right image, and respectively calculate the angles corresponding to each vertex in the left image and the right image;

[0062] For these two images, the corner coordinates of the four vertices of the chessboard grid will be extracted first. After extraction, the straight-line distances between the respective corner points can be calculated.

[0063] As Figure 3 shown, for the above left image or right image, the rectangle generated after extracting the corner points A, B, C, and D on the chessboard. Due to different shooting angles, an angle that is actually 90 degrees may not be 90 degrees in the image. Therefore, according to the angles corresponding to each corner point, the corresponding shooting direction can be obtained.

[0064] Specifically, after knowing the coordinates of A, B, C, and D, the distances between the respective corner points can be calculated, and according to Figure 3 it can be known that there are triangles ABC, BCD, ADC, and ABD. Therefore, according to the cosine theorem, the angles DAB, ABC, BCD, and CDA can be respectively calculated, and for the sake of simplicity of description, they are respectively abbreviated as angle A, angle B, angle C, and angle D, corresponding to Figure 3 the θ1, θ2, θ3, and θ4 marked in

[0065] Since there are the left figure and the right figure, the angles of a total of 8 corners can be calculated. The four corners of the right figure are denoted as θ r1 , θ r2 , θ r3 , θ r4 , and the four corners of the left figure are denoted as θ l1 , θ l2 , θ l3 , θ l4 . The four corners of the left figure and the right figure respectively correspond to the above-mentioned angles A, B, C, and D in this embodiment.

[0066] Step S320: Screen out the angles that meet the conditions, and judge the matching relationship between the pose of the camera and the preset direction according to whether the screened angles are within the preset angle range, where different preset angle ranges correspond one-to-one to different preset directions.

[0067] After obtaining the above angles, the differences between these angles and the right angle can be calculated respectively. For the angles formed by the same vertex, select the angle formed by the vertex with a larger difference as the angle that meets the conditions.

[0068] Specifically, the screening conditions are shown in the following expression:

[0069]

[0070] where i = 1, 2, 3, 4

[0071] θ i is the finally screened angle. In this embodiment, 4 angles will be finally screened out.

[0072] According to the angles of these 4 angles, it can be judged which one of the above 9 directions the shooting direction of the binocular camera corresponds to.

[0073] Specifically, there are the following judgment formulas to judge the shooting direction.

[0074] Straight ahead: -δ1 < θ1, θ2, θ3, θ4 < δ1

[0075] Directly below: δ2 > θ1, θ2 > δ1 && -δ2 < θ3, θ4 < -δ1

[0076] Directly above: -δ2 < θ1, θ2 < -δ1 && δ2 > θ3, θ4 > δ1

[0077] Directly to the left: δ2 > θ2, θ4 > δ1 && -δ2 < θ1, θ3 < -δ1

[0078] Directly to the right: δ2 > θ1, θ3 > δ1 && -δ2 < θ2, θ4 < -δ1

[0079] Lower left corner: -δ1 < θ1, θ4 < δ1, δ2 > δ2 > δ1, -δ2 < θ3 < -δ1

[0080] Upper left corner: -δ1 < θ2, θ3 < δ1, δ2 > θ4 > δ1, -δ2 < θ1 < -δ1

[0081] Lower right corner: -δ1 < θ2, θ3 < δ1, δ2 > θ1 > δ1, -δ2 < θ4 < -δ1

[0082] Upper right corner: -δ1 < θ1, θ4 < δ1, δ2 > θ3 > δ1, -δ2 < θ2 < -δ1

[0083] Where δ1 and δ2 are preset thresholds, and according to the actual test results, δ1 < δ2.

[0084] In this embodiment, the set δ1 and δ2 are bounded by prior tests and debugging, and then set accordingly according to the actual application scenario. Their unit is degree. For example, if δ1 = 4 and δ2 = 6, it means δ1 is 4 degrees and δ2 is 6 degrees.

[0085] If the four angles satisfy one of the above 9 conditions, it means it conforms to the preset direction. If not, it means the shooting direction of the current binocular camera is not the preset direction.

[0086] Step S400, if the shooting direction of the chessboard does not conform to the preset direction, adjust the robotic arm until the shooting direction of the binocular camera conforms to the preset direction and then take a picture.

[0087] When it does not conform to the preset direction, the robotic arm needs to be adjusted. It can be understood that if the shooting direction of the chessboard exactly conforms to the preset direction, just take a picture directly.

[0088] Taking a 5-axis robotic arm as an example, it has 2 yaw axes, 2 pitch axes and 1 roll axis. The yaw axis controls the left and right head turning, the pitch axis controls the up and down nodding, and the roll axis controls the rotation. Among them, yaw axis 1 realizes large-angle switching in the left and right directions, and yaw axis 2 realizes fine adjustment in the left and right directions. Pitch axis 1 realizes large-angle switching in the up and down directions, and pitch axis 2 realizes fine adjustment of the pitch angle. The roll axis realizes fine adjustment of the rotation.

[0089] (1) When taking a picture directly below the chessboard grid, it is necessary to achieve δ2 > θ1, θ2 > δ1 && -δ2 < θ3, θ4 < -δ1. If θ1, θ2 < δ1, it is necessary to finely adjust the pitch axis to increase the downward-looking angle of the binocular camera shooting angle.

[0090] If θ1, θ2 > δ2, it is necessary to finely adjust the pitch axis to decrease the downward-looking angle of the binocular camera shooting angle.

[0091] The angle adjustments of θ3 and θ4 are the same. Therefore, they will not be elaborated here.

[0092] (2) When taking a photo directly above the checkerboard, it is necessary to achieve -δ2 < θ1, θ2 < -δ1 && δ2 > θ3, θ4 > δ1

[0093] If θ1, θ2 > -δ1, then it is necessary to finely adjust the pitch axis to increase the upward looking angle of the binocular camera.

[0094] If θ1, θ2 > δ2, then it is necessary to finely adjust the pitch axis to decrease the upward looking angle of the binocular camera. The angle adjustments of θ3 and θ4 are the same.

[0095] (3) When taking a photo directly to the left of the checkerboard, it is necessary to achieve δ2 > θ2, θ4 > δ1 && -δ2 < θ1, θ3 < -δ1

[0096] If θ2, θ4 < δ1, then it is necessary to finely adjust the yaw axis to make the binocular camera yaw to the right. If θ2, θ4 > δ2, then it is necessary to finely adjust the yaw axis to make the binocular camera yaw to the left. The angle adjustments of θ1 and θ3 are the same. Therefore, they will not be elaborated here.

[0097] The angle adjustments of θ1 and θ3 are the same. Therefore, they will not be elaborated here.

[0098] (4) When taking a photo directly to the right of the checkerboard, it is necessary to achieve δ2 > θ1, θ3 > δ1 && -δ2 < θ2, θ4 < -δ1.

[0099] If θ1, θ3 < δ1, then it is necessary to finely adjust the yaw axis to make the binocular camera yaw to the left. If θ1, θ3 > δ2, then it is necessary to finely adjust the yaw axis to make the binocular camera yaw to the right. The angle adjustments of θ3 and θ4 are the same. Therefore, they will not be elaborated here.

[0100] The angle adjustments of θ3 and θ4 are the same. Therefore, they will not be elaborated here.

[0101] (5) When taking a photo in the lower left of the checkerboard, it is necessary to achieve -δ1 < θ1, θ4 < δ1, δ2 > θ2 > δ2, -δ2 < θ3 < -δ1.

[0102] The pitch axis 1 adjusts the binocular camera at a large angle below the checkerboard, and then the yaw axis 1 adjusts the binocular camera at a large angle to the left of the checkerboard. Then finely adjust the pitch axis 2 to make the binocular camera look slightly upward, and then adjust the yaw axis 2 to make the binocular camera look slightly to the right.

[0103] At this time, if θ2 < δ1 or θ3 > -δ1, then it is necessary to finely adjust the yaw axis 2 to the right and finely adjust the pitch axis 2 upward.

[0104] If θ2 > δ2 or θ3 < -δ2, then it is necessary to finely adjust the yaw axis 2 to the left and finely adjust the pitch axis 2 downward.

[0105] (6) When taking a photo from the upper left of the checkerboard, it is necessary to achieve -δ1 < θ2, θ3 < δ1, δ2 > θ4 > δ1, -δ2 < θ1 < -δ1.

[0106] The pitch axis 1 adjusts the binocular camera at a large angle above the checkerboard, and then the yaw axis 1 adjusts the binocular camera at a large angle to the left of the checkerboard. Then, finely adjust the pitch axis 2 to make the binocular camera look slightly downward, and then adjust the yaw axis 2 to make the binocular camera move slightly to the right.

[0107] At this time, if θ4 < δ1 or θ1 > -δ1, it is necessary to finely adjust the yaw axis 2 to the right and the pitch axis 2 downward simultaneously.

[0108] If θ4 > δ2 or θ1 < -δ2, it is necessary to finely adjust the yaw axis 2 to the left and the pitch axis 2 upward simultaneously.

[0109] (7) When taking a photo from the lower right of the checkerboard, it is necessary to achieve -δ1 < θ2, θ3 < δ1, δ2 > θ1 > δ1, -δ2 < θ4 < -δ1.

[0110] The pitch axis 1 adjusts the binocular camera at a large angle below the checkerboard, and then the yaw axis 1 adjusts the binocular camera at a large angle to the right of the checkerboard. Then, finely adjust the pitch axis 2 to make the binocular camera look slightly upward, and then adjust the yaw axis 2 to make the binocular camera move slightly to the left.

[0111] At this time, if θ1 < δ1 or θ4 > -δ1, it is necessary to finely adjust the yaw axis 2 to the left and the pitch axis 2 upward simultaneously.

[0112] If θ1 > δ2 or θ4 < -δ2, it is necessary to finely adjust the yaw axis 2 to the right and the pitch axis 2 downward simultaneously.

[0113] (8) When taking a photo from the upper right of the checkerboard, it is necessary to achieve -δ1 < θ1, θ4 < δ1, δ2 > θ3 > δ1, -δ2 < θ2 < -δ1.

[0114] The pitch axis 1 adjusts the binocular camera at a large angle above the checkerboard, and then the yaw axis 1 adjusts the binocular camera at a large angle to the right of the checkerboard. Then, finely adjust the pitch axis 2 to make the binocular camera look slightly downward, and then adjust the yaw axis 2 to make the binocular camera move slightly to the left.

[0115] At this time, if θ2 < δ1 or θ3 > -δ1, it is necessary to finely adjust the yaw axis 2 to the left and the pitch axis 2 downward simultaneously.

[0116] If θ2 > δ2 or θ3 < -δ2, it is necessary to finely adjust the yaw axis 2 to the right and the pitch axis 2 upward simultaneously.

[0117] After the adjustment, the photographing operation can be performed. Since there are multiple directions to be photographed, the robotic arm will initially take pictures in sequence according to the program settings for each preset direction. Each time a picture is taken in a preset direction, that direction is marked as completed, and it moves to the next preset direction for photographing. When all the preset directions are marked as completed, the photographing ends.

[0118] Specifically, a photographing completion flag bit can be set. Initially, the flag bits for all preset directions are 0. After taking a picture in a direction, the corresponding direction's flag bit is set to 1. This can prevent repeated photographing of the same position and also facilitate checking for missed shots, so as not to affect the calibration success rate and accuracy of the binocular camera.

[0119] Embodiment 2

[0120] This application embodiment also provides a binocular camera calibration control device, as Figure 5 shown, including:

[0121] The first judgment module 10 is used to obtain the image data of the checkerboard image photographed by the binocular camera, and determine whether the checkerboard photographed by the binocular camera is complete according to the image data;

[0122] The first adjustment module 20 is used to adjust the robotic arm if the checkerboard in the image data is incomplete, so that the binocular camera can photograph a complete checkerboard image;

[0123] The second judgment module 30 is used to judge whether the photographing direction of the binocular camera conforms to the preset direction according to the image data;

[0124] The second adjustment module 40 is used to adjust the robotic arm until the photographing direction of the binocular camera conforms to the preset direction for photographing if the photographing direction of the checkerboard does not conform to the preset direction.

[0125] This application embodiment also provides a control device, including a processor and a memory. The memory stores a computer program, and when the computer program runs on the processor, it executes the binocular camera calibration control method.

[0126] This application embodiment also provides a readable storage medium, which stores a computer program, and when the computer program runs on a processor, it executes the binocular camera calibration control method.

[0127] In several embodiments provided in this application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and structural diagrams in the accompanying drawings show the possible architectures, functions, and operations of devices, methods, and computer program products according to multiple embodiments of the present invention. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a part of code, and the module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in an alternative implementation, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks can actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the structural diagram and / or flowchart, as well as the combination of blocks in the structural diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0128] In addition, each functional module or unit in various embodiments of the present invention can be integrated together to form an independent part, or each module can exist alone, or two or more modules can be integrated to form an independent part.

[0129] If the above functions are implemented in the form of software functional modules and sold or used as an independent product, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a smart phone, a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0130] The above is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should all be covered by the protection scope of the present invention.

Claims

1. A binocular camera calibration control method, characterized in that, The binocular camera is installed on the robotic arm for calibration, and the method includes: Obtain the image data of the checkerboard image captured by the binocular camera, and determine whether the checkerboard captured by the binocular camera is complete according to the image data; If the checkerboard in the image data is incomplete, adjust the robotic arm so that the binocular camera captures a complete checkerboard image; Judge whether the shooting direction of the binocular camera conforms to the preset direction according to the image data; If the shooting direction of the checkerboard does not conform to the preset direction, adjust the robotic arm until the shooting direction of the binocular camera conforms to the preset direction and then perform shooting; The image data of the checkerboard image captured by the binocular camera includes the left image captured by the left camera of the binocular camera and the right image captured by the right camera of the binocular camera; The judging whether the shooting direction of the binocular camera conforms to the preset direction according to the image data includes: Extract the coordinates of the four vertices of the checkerboard in the left image and the right image respectively, and calculate the angle corresponding to each vertex in the left image and the right image respectively; Screen out the qualified angles, and judge the matching relationship between the pose of the camera and the preset direction according to whether the screened angles are within the preset angle range, where different preset angle ranges correspond one-to-one to different preset directions.

2. The binocular camera calibration control method according to claim 1, characterized in that, It also includes: When shooting, every time shooting is performed in one of the preset directions, mark that direction as completed, and move to the next preset direction for shooting. When all preset directions are marked as completed, end the shooting.

3. The binocular camera calibration control method according to claim 1, wherein Adjusting the robotic arm until the shooting direction of the binocular camera conforms to the preset direction includes: Determine the relative position relationship between the pose of the camera and the preset direction according to the size relationship between the screened angles and the preset angle range; According to the relative position relationship, control the robotic arm to perform corresponding left-right translation, up-down translation or rotation so that the camera pose conforms to the preset direction.

4. The binocular camera calibration control method according to claim 1, characterized in that Screening out the qualified angles includes: Compare the difference between the angle formed by the same vertex in the left image and the right image and the right angle respectively, and select the angle formed by the vertex with the larger difference as the qualified angle.

5. The binocular camera calibration control method according to claim 1, wherein The preset direction is any one of directly in front of the checkerboard, directly to the left, directly to the right, directly above, directly below, upper left, upper right, lower left and lower right.

6. The binocular camera calibration control method according to claim 1, wherein, If the checkerboard in the image data is incomplete, adjusting the robotic arm includes: Calculate whether the area of the checkerboard in the image data matches the checkerboard. If not, the checkerboard is incomplete, and then control the robotic arm to move the binocular camera until the checkerboard in the image data is complete.

7. A binocular camera calibration control device, characterized in that It includes: The first judgment module is used to obtain the image data of the checkerboard image captured by the binocular camera, and determine whether the checkerboard captured by the binocular camera is complete according to the image data; The first adjustment module is used to adjust the robotic arm if the checkerboard in the image data is incomplete, so that the binocular camera can capture a complete checkerboard image; The second judgment module is used to judge whether the shooting direction of the binocular camera conforms to the preset direction according to the image data; A second adjustment module, configured to adjust the robotic arm until the shooting direction of the binocular camera conforms to the preset direction for shooting if the shooting direction of the chessboard does not conform to the preset direction; The image data of the chessboard image captured by the binocular camera includes the left image captured by the left camera of the binocular camera and the right image captured by the right camera of the binocular camera; The determination of whether the shooting direction of the binocular camera conforms to the preset direction according to the image data includes: Respectively extract the coordinates of the four vertices of the chessboard in the left image and the right image, and respectively calculate the angles corresponding to each vertex in the left image and the right image; Screen out the qualified angles, and determine the matching relationship between the pose of the camera and the preset direction according to whether the screened angles are within the preset angle intervals, where different preset angle intervals have a one-to-one correspondence with different preset directions.

8. A terminal device, characterized in that, It includes a processor and a memory, and the memory stores a computer program, and the computer program executes the binocular camera calibration control method according to any one of claims 1 to 6 when running on the processor.

9. A readable storage medium, characterized in that, It stores a computer program, and the computer program executes the binocular camera calibration control method according to any one of claims 1 to 6 when running on the processor.

Images