A camera calibration method, apparatus, device, system and storage medium

Abstract

The application provides a camera calibration method, device, equipment, system and storage medium, and relates to the field of camera calibration. The application acquires a serial number of a camera to be calibrated, and determines a motion trajectory corresponding to the serial number. In a preset acquisition time period, a mechanical arm is controlled to drive the camera to move according to the motion trajectory corresponding to the serial number, and the camera is controlled to acquire calibration images by shooting a calibration board in the moving process. The camera is installed on the mechanical arm. The calibration images acquired by the camera are acquired, and the calibration images are recorded according to the serial number. Since the calibration images acquired by the embodiment of the application are acquired in the moving process of the camera, the embodiment of the application determines calibration images according to the calibration images acquired in the moving process of the camera, and determines calibration parameters of different cameras, so that the cost of camera calibration is reduced.

Description

Technical Field

[0001] This application relates to the field of camera calibration, specifically to a camera calibration method, apparatus, device, system, and storage medium. Background Technology

[0002] With the advancement of science and technology, image processing technology has begun to play a significant role in various fields, including public transportation, agricultural technology, and aviation. As image technology develops, the requirements for image accuracy are also increasing. In the field of artificial intelligence, obtaining robot position information in its environment through camera captures also requires meticulous processing of image data, especially considering the significant impact of minute deviations in the optical center and distortion during camera manufacturing.

[0003] Currently, cameras include monocular cameras, multi-view cameras, and IMU cameras equipped with inertial measurement units (IMUs). Existing camera calibration methods do not support calibration for different types of cameras, nor are they suitable for efficient batch calibration of cameras to obtain calibration parameters; therefore, existing calibration methods are not only costly but also inefficient. Summary of the Invention

[0004] To address the existing technical problems, embodiments of this application provide a camera calibration method, apparatus, device, system, and storage medium to reduce the cost of camera calibration.

[0005] To achieve the above objectives, the technical solution of this application embodiment is implemented as follows:

[0006] In a first aspect, embodiments of this application provide a camera calibration method, the method comprising:

[0007] Obtain the serial number of the camera to be calibrated, and determine the motion trajectory corresponding to the serial number;

[0008] During a preset data acquisition period, the robotic arm is controlled to move the camera along the motion trajectory corresponding to the serial number, and the camera is controlled to acquire calibration images by shooting the calibration plate during the movement; the camera is mounted on the robotic arm.

[0009] Acquire the calibration image captured by the camera and record the calibration image according to the serial number.

[0010] Because this embodiment of the application obtains the camera's serial number, determines the motion trajectory corresponding to the serial number, and controls the robotic arm to drive the camera to move according to the motion trajectory corresponding to the serial number during a preset acquisition time period, and controls the camera to acquire calibration images by shooting the calibration board during the movement; then, it acquires the calibration images acquired by the camera and records the calibration images according to the serial number, so that this embodiment of the application can determine the calibration parameters of different types of cameras based on the calibration images acquired by the camera in motion, thereby reducing the calibration cost.

[0011] An optional implementation further includes, after acquiring the calibration image captured by the camera and recording the calibration image according to the serial number, the method further includes:

[0012] During a preset replacement time period, the camera mounted on the robotic arm is replaced with a camera that has not acquired calibration images, and the serial number of the replaced camera is obtained. The motion trajectory corresponding to the replaced camera is determined based on the serial number. The replacement time period and the acquisition time period alternate periodically and continuously.

[0013] Since this embodiment of the application, after obtaining the calibration image captured by the camera, replaces the camera mounted on the robotic arm with a camera that has not captured calibration images during a preset replacement time period, and determines the motion trajectory corresponding to the replaced camera based on the obtained serial number of the replaced camera, this embodiment of the application can obtain the calibration image captured by the replaced camera during the motion process at the end of the replacement time period and the beginning of the acquisition time period, thereby improving the acquisition efficiency of calibration images.

[0014] An optional implementation further includes, after acquiring the calibration image captured by the camera and recording the calibration image according to the serial number, the method further includes:

[0015] The calibration image is sent to the calibration device so that the calibration device can determine the calibration parameters corresponding to the camera based on the calibration image.

[0016] Since the embodiments of this application obtain the calibration image captured by the camera, record the calibration image according to the serial number, and then send the calibration image to the calibration device, the calibration device determines the corresponding calibration parameters of the camera, thereby improving the calibration efficiency.

[0017] An optional implementation further includes, before obtaining the camera's serial number and determining the motion trajectory corresponding to the serial number, the method further comprising:

[0018] Based on the size of the calibration plate, the field of view of the camera, and the distance between the camera and the calibration plate, determine the position of the robotic arm when the calibration plate appears at the lower left, lower right, upper left, upper right, and center positions of the camera's field of view, respectively.

[0019] The range of motion of the robotic arm is determined based on its position.

[0020] Because this embodiment determines the position of the robotic arm when the calibration plate appears at the lower left, lower right, upper left, upper right, and center positions of the camera's field of view based on the size of the calibration plate, the camera's field of view, and the distance between the camera and the calibration plate; and then determines the range of motion of the robotic arm based on the position of the robotic arm; and the robotic arm moves according to a preset trajectory within the determined range of motion, so that the calibration images acquired by the camera during the movement all contain complete images of the calibration plate, thereby improving the calibration accuracy when determining calibration parameters based on calibration images containing complete images of the calibration plate.

[0021] Secondly, embodiments of this application provide a camera calibration device, comprising:

[0022] A determining unit is used to obtain the serial number of the camera to be calibrated and determine the motion trajectory corresponding to the serial number;

[0023] The control unit is used to control the robotic arm to drive the camera to move according to the motion trajectory corresponding to the serial number during a preset acquisition time period, and to control the camera to acquire calibration images by shooting the calibration plate during the movement; the camera is mounted on the robotic arm;

[0024] The acquisition unit is used to acquire the calibration image captured by the camera and record the calibration image according to the serial number.

[0025] In one optional implementation, after acquiring the calibration image captured by the camera and recording the calibration image according to the sequence number, the determining unit is specifically used for:

[0026] During a preset replacement time period, the camera mounted on the robotic arm is replaced with a camera that has not acquired calibration images, and the serial number of the replaced camera is obtained. The motion trajectory corresponding to the replaced camera is determined based on the serial number. The replacement time period and the acquisition time period alternate periodically and continuously.

[0027] In one optional implementation, after acquiring the calibration image captured by the camera and recording the calibration image according to the serial number, the acquisition unit is further configured to:

[0028] The calibration image is sent to the calibration device so that the calibration device can determine the calibration parameters corresponding to the camera based on the calibration image.

[0029] In one optional implementation, before obtaining the camera's serial number and determining the motion trajectory corresponding to the serial number, the determining unit is further configured to:

[0030] Based on the size of the calibration plate, the field of view of the camera, and the distance between the camera and the calibration plate, determine the position of the robotic arm when the calibration plate appears at the lower left, lower right, upper left, upper right, and center positions of the camera's field of view, respectively.

[0031] The range of motion of the robotic arm is determined based on its position.

[0032] Thirdly, embodiments of this application provide a control device, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and when the computer program is executed by the processor, the processor causes the processor to implement the camera calibration method of any of the first aspects described above.

[0033] Fourthly, embodiments of this application provide a camera calibration system, including a control device, a calibration device, a robotic arm, and a camera;

[0034] The control device is used to acquire the serial number of the camera to be calibrated and determine the motion trajectory corresponding to the serial number; during a preset acquisition time period, it controls a robotic arm to drive the camera to move according to the motion trajectory corresponding to the serial number, and controls the camera to acquire calibration images by shooting a calibration plate during the movement; it acquires the calibration images acquired by the camera and records the calibration images according to the serial number; it sends the calibration images to the calibration device, and during a preset replacement time period, it replaces the camera mounted on the robotic arm with a camera that has not acquired calibration images, acquires the serial number of the replaced camera, and determines the motion trajectory corresponding to the replaced camera according to the serial number; the replacement time period and the acquisition time period alternate periodically and continuously.

[0035] The robotic arm is used to drive the camera to move according to the motion trajectory corresponding to the serial number;

[0036] The camera is used to capture calibration images of the calibration plate during movement;

[0037] The calibration device is used to determine the calibration parameters corresponding to the camera based on the calibration image sent by the control device; if the calibration parameters are found to be unqualified, the camera corresponding to the calibration parameters is calibrated a second time; if the calibration parameters are found to be qualified, the calibration parameters are stored according to the serial number.

[0038] Fifthly, embodiments of this application provide a computer storage medium that stores computer program instructions. When the instructions are executed on a computer, the computer performs any of the camera calibration methods described in the first aspect above.

[0039] The technical effects of any of the implementation methods in aspects two through five can be found in the technical effects of the corresponding implementation methods in aspect one, and will not be repeated here. Attached Figure Description

[0040] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 A schematic flowchart illustrating a camera calibration method provided in an embodiment of this application;

[0042] Figure 2 A schematic diagram of a timing control diagram provided in an embodiment of this application;

[0043] Figure 3 A complete flowchart of a camera calibration method provided in this application embodiment;

[0044] Figure 4 This is a schematic diagram of a camera calibration system provided in an embodiment of this application;

[0045] Figure 5 An interactive information diagram illustrating a camera calibration method provided in an embodiment of this application;

[0046] Figure 6 A structural block diagram of a camera calibration device provided in an embodiment of this application;

[0047] Figure 7 This is a schematic diagram of the structure of a control device provided in an embodiment of this application. Detailed Implementation

[0048] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0049] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0050] Currently, existing camera calibration methods can only calibrate one type of camera and cannot calibrate cameras in batches efficiently to determine the camera calibration parameters.

[0051] Based on the above issues, such as Figure 1 As shown in the figure, this application provides a camera calibration method applied to a control device, including the following steps:

[0052] Step S101: Obtain the serial number of the camera to be calibrated and determine the motion trajectory corresponding to the serial number.

[0053] In some embodiments, the control device can obtain the serial number of the camera to be calibrated by scanning the camera to be calibrated.

[0054] In other embodiments, the present application embodiments may scan the camera to be calibrated using other devices, determine the serial number of the camera to be calibrated, and then send the determined serial number to the control device.

[0055] In step S102, during the preset acquisition time period, the robotic arm is controlled to drive the camera to move along the motion trajectory corresponding to the serial number, and the camera is controlled to acquire calibration images by shooting the calibration plate during the movement; the camera is mounted on the robotic arm.

[0056] In some embodiments, during a preset acquisition time period, the control device can send a motion command containing a motion trajectory to the robotic arm to control the robotic arm to move the camera; the robotic arm moves according to the preset motion trajectory based on the received motion command, and the camera mounted on the robotic arm moves with the robotic arm; and, at the same time as sending the motion command to the robotic arm, the control device sends an image acquisition command to the camera to control the camera to turn on and capture calibration images of the calibration plate during the movement.

[0057] Step S103: Acquire the calibration image captured by the camera and record the calibration image according to the serial number.

[0058] In some embodiments, after acquiring the calibration image captured by the camera, the control device records the acquired calibration image according to the camera's serial number.

[0059] Because this embodiment of the application obtains the camera's serial number, determines the motion trajectory corresponding to the serial number, and controls the robotic arm to drive the camera to move according to the motion trajectory corresponding to the serial number during a preset acquisition time period, and controls the camera to acquire calibration images by shooting the calibration board during the movement; then, it acquires the calibration images acquired by the camera and records the calibration images according to the serial number, so that this embodiment of the application can determine the calibration parameters of different types of cameras based on the calibration images acquired by the camera in motion, thereby reducing the calibration cost.

[0060] In some embodiments, before obtaining the serial number of the camera to be calibrated and determining the motion trajectory corresponding to the serial number, the control device determines the motion range of the robotic arm and then determines the motion trajectory of the robotic arm based on the motion range of the robotic arm.

[0061] In specific implementation, the movement range of the robotic arm can be determined in the following ways according to the embodiments of this application:

[0062] In some embodiments, the control device determines the position of the robotic arm when the calibration plate appears at the lower left, lower right, upper left, upper right, and center positions of the camera's field of view, based on the size of the calibration plate, the camera's field of view, and the distance between the camera and the calibration plate; the control device then determines the robotic arm's range of motion based on the determined range of motion.

[0063] In one embodiment, the present application embodiment can drive the camera to move by fixing the calibration plate and moving the robotic arm, so that the position of the calibration plate changes when the camera captures the calibration plate. The position point to which the robotic arm moves is determined when the calibration plate appears at the lower left corner, lower right corner, upper left corner, upper right corner and center position of the camera's field of view; and then the range of motion of the robotic arm is determined based on the determined position point of the robotic arm.

[0064] In another embodiment, this application embodiment fixes the camera and moves the calibration plate to determine the position points of the calibration plate when it appears at the lower left, lower right, upper left, upper right, and center positions of the camera's field of view; based on the positional correspondence between the calibration plate and the camera, the corresponding position point of the camera when the calibration plate is fixed is determined; furthermore, based on the position of the camera mounted on the robotic arm and the determined position point of the camera, the position point of the robotic arm is determined; and based on the determined position point of the robotic arm, the range of motion of the robotic arm is determined.

[0065] In practice, the control equipment can determine the three-dimensional space formed by the determined position points of the robotic arm, and use the determined three-dimensional space as the range of motion of the robotic arm.

[0066] In this embodiment, the camera can capture the calibration plate completely within the range of motion of the robotic arm. That is, when the camera moves within the range of motion of the robotic arm, the calibration plate is always completely within the camera's field of view.

[0067] It should be noted that, in the embodiments of this application, the range of motion of the robotic arm can be determined by a control device or by other devices, and no limitation is made here.

[0068] In some embodiments, after determining the range of motion of the robotic arm, the control device determines the motion trajectory corresponding to the serial number of the camera to be calibrated.

[0069] In some embodiments, the control device records the movement trajectory of the robotic arm based on the trajectory information during the movement of the robotic arm.

[0070] It should be noted that the trajectory information includes both location and angle information.

[0071] In one embodiment, the control device selects multiple trajectory position points around the five determined position points of the robotic arm, controls the robotic arm to move to each trajectory position point in sequence, and records the movement trajectory of the robotic arm as movement trajectory A.

[0072] It should be noted that the trajectory positions are within the range of motion of the robotic arm. The camera takes pictures of the calibration board at multiple trajectory positions, acquiring calibration images; the pixels representing the calibration board in the multiple calibration images acquired by the camera can completely cover all pixels in the calibration images.

[0073] In another embodiment, the control device uses the position of the robotic arm when the calibration plate is at the center of the camera's field of view as a reference position point, controls the robotic arm to perform continuous movements, and records the movement trajectory of the robotic arm as movement trajectory B. For example, the continuous movement of the robotic arm can be around the reference position point by moving up, down, left, right, forward, or backward, or by rotating around the X, Y, and Z axes of the robotic arm, or by performing a figure-eight movement around the reference position point.

[0074] It should be noted that the camera is always able to capture the calibration plate completely while the robotic arm is moving continuously.

[0075] In some embodiments, the control device selects multiple trajectory position points around the five determined position points of the robotic arm, controls the robotic arm to move sequentially to each trajectory position point, and records the movement trajectory of the robotic arm as the movement trajectory C.

[0076] It should be noted that the trajectory C can be the same as or different from the trajectory A; no restrictions are imposed here.

[0077] In this embodiment, the time taken for the robotic arm to move along trajectory A is the same as the time taken for the robotic arm to move along trajectories B and C, and is also the same as the preset data collection time period.

[0078] In some embodiments, the control device establishes a correspondence table between the recorded motion trajectory of the robotic arm and the serial number of the camera, and sends the determined motion trajectory to the robotic arm so that the robotic arm can move according to the pre-stored motion trajectory after receiving the motion command.

[0079] In some embodiments, after obtaining the serial number of the camera to be calibrated, the control device determines the motion trajectory corresponding to the serial number according to a preset correspondence table between serial numbers and motion trajectories.

[0080] It should be noted that different camera models correspond to different motion trajectories, while cameras of the same model correspond to the same motion trajectory.

[0081] For example, the control device determines the motion trajectory corresponding to the sequence number according to a preset correspondence table between sequence numbers and motion trajectories, as shown in Table 1:

[0082] Table 1. Correspondence between Serial Number and Motion Trajectory

[0083]

[0084] In Table 1, D-001~DN represent the serial numbers of multi-view cameras; S-001~SN represent the serial numbers of monocular cameras; and I-001~IN represent the serial numbers of IMU cameras.

[0085] In some embodiments, after determining the motion trajectory corresponding to the camera's serial number, the control device controls the robotic arm to move the camera according to the motion trajectory corresponding to the serial number during a preset acquisition time period, and controls the camera to acquire calibration images by shooting the calibration plate during the movement.

[0086] In specific implementation, the camera in this application embodiment can acquire calibration images in the following ways:

[0087] In some embodiments, when motion trajectory A is a smooth and continuous motion trajectory of the robotic arm, the camera continuously acquires calibration images as it moves along motion trajectory A under the drive of the robotic arm.

[0088] In other embodiments, when the motion trajectory A is a motion trajectory in which the robotic arm moves sequentially to each trajectory position point and stops for a preset time, the camera, driven by the robotic arm, moves along the motion trajectory A and acquires a calibration image within a preset time after reaching the trajectory position point.

[0089] In some embodiments, the camera continuously acquires calibration images as it moves along motion trajectory B under the drive of the robotic arm.

[0090] In this embodiment of the application, the control device controls the camera to acquire calibration images, obtains the calibration images acquired by the camera, and records the acquired calibration images according to the camera's serial number.

[0091] In some embodiments, when the motion trajectory corresponding to the serial number is motion trajectory A, the control device acquires the calibration image captured by the camera and records the acquired calibration image as a calibration dataset according to the serial number.

[0092] For example, when the camera's serial number is S-001, and the motion trajectory corresponding to S-001 is determined to be motion trajectory A, the control device will record the acquired calibration image as the S-001-A calibration dataset.

[0093] In other embodiments, when the motion trajectory corresponding to the camera's serial number is motion trajectory B or C, the control device acquires sensor data of the camera during the motion process while acquiring the calibration image captured by the camera, and records the acquired calibration image and sensor data as a calibration dataset according to the serial number.

[0094] It should be noted that the control device controls the robotic arm to move in the order of motion trajectories B and C; or the control device controls the robotic arm to move in the order of motion trajectories C and B.

[0095] For example, when the control device calibrates the IMU camera, after determining that the IMU camera's serial number is I-001, it determines the motion trajectory corresponding to I-001 as motion trajectory B and C according to the correspondence table between the serial number and the motion trajectory. The control device records the calibration image I-001-C calibration dataset obtained by the camera when controlling the robotic arm to move according to motion trajectory C, and the calibration image and sensor data obtained by the camera when controlling the robotic arm to move according to motion trajectory B as the I-001-B calibration dataset.

[0096] In some embodiments, after acquiring calibration images captured by the camera and recording the calibration images as a calibration dataset, the control device sends the calibration dataset to the calibration device. During a preset replacement time period, the control device replaces the camera mounted on the robotic arm with a camera that has not acquired calibration images, obtains the serial number of the replaced camera, and determines the motion trajectory corresponding to the replaced camera based on the serial number.

[0097] In this embodiment of the application, after acquiring the calibration image and recording the calibration image according to the serial number, the control device simultaneously performs the two operations described above. The two operations are described below.

[0098] Operation 1: The control device sends the acquired calibration image to the calibration device.

[0099] In some embodiments, after acquiring the calibration image captured by the camera and recording the calibration image according to the serial number, the control device sends the calibration image to the calibration device so that the calibration device can determine the calibration parameters corresponding to the camera based on the calibration image.

[0100] In practice, after acquiring the calibration images captured by the camera and recording them as a calibration dataset, the control device sends the calibration dataset to the calibration device.

[0101] In some embodiments, after receiving the calibration dataset, the calibration device determines the calibration parameters corresponding to the camera based on the calibration dataset.

[0102] For example, after receiving the S-001-A calibration dataset, the calibration device determines the calibration parameters corresponding to the monocular camera based on the calibration images in the calibration dataset.

[0103] It should be noted that the calibration parameters include both the camera's internal and external parameters.

[0104] In some embodiments, the calibration device determines the camera's internal parameters based on the calibration images in the received calibration dataset; then, it determines the camera's external parameters based on the determined internal parameters and the calibration images.

[0105] For example, after receiving the D-001-A calibration dataset, the calibration device determines the internal parameters corresponding to the stereo camera based on the calibration images in the calibration dataset; based on the calibration images, it determines the images of the overlapping fields of view of the cameras in the stereo camera; and then, based on the determined internal parameters, it determines the external parameters between the cameras of the stereo camera.

[0106] In other embodiments, the calibration device determines the camera's internal parameters based on the calibration images in the received calibration dataset; the calibration device determines the camera's sensor parameters based on the serial number corresponding to the received calibration dataset; and the calibration device determines the camera's external parameters based on the determined camera's internal parameters and sensor parameters.

[0107] For example, after receiving the I-001-A calibration dataset and the I-001-B calibration dataset, the calibration device determines the internal parameters of the IMU camera based on the calibration images in the I-001-A calibration dataset; then, based on the determined internal parameters and the calibration images and IMU sensor parameters in the I-001-B calibration dataset, it determines the external parameters of the IMU camera, such as the relative positional relationship between the camera and the IMU in the IMU camera, the visual optical center coordinate system of the IMU camera, and the external parameters of the IMU coordinate system.

[0108] It should be noted that the calibration device can process calibration images serially to determine the calibration parameters corresponding to the camera, or it can process calibration images in parallel to determine the calibration parameters corresponding to the camera. This application embodiment does not impose any limitations on this.

[0109] In some embodiments, after determining the calibration parameters corresponding to the camera, the calibration device performs testing on the determined calibration parameters.

[0110] In practice, calibration equipment can test calibration parameters in the following ways:

[0111] In some embodiments, the calibration device determines the projection error corresponding to the camera based on the determined calibration parameters; if the calibration device determines that the projection error is less than or equal to a preset threshold, the calibration parameters are deemed qualified; if the calibration device determines that the projection error is greater than the preset threshold, the calibration parameters are deemed unqualified.

[0112] In some embodiments, after the calibration device detects that the calibration parameters are qualified, it stores the calibration parameters according to the camera's serial number.

[0113] In this embodiment of the application, calibration parameters can be input into the camera through the camera's control program, or the camera's calibration parameters can be written into the camera's hardware.

[0114] In other embodiments, after detecting that the calibration parameters are unqualified, the calibration device sends a notification to the control device; after receiving the notification from the calibration device, the control device performs multiple calibrations on the camera corresponding to the unqualified calibration parameters. If it is determined that the multiple calibration parameters are unqualified, the camera is determined to be an unqualified camera.

[0115] Operation 2: Control the equipment to replace the camera.

[0116] In some embodiments, the control device replaces the camera mounted on the robotic arm with a camera that has not acquired calibration images during a preset replacement time period, obtains the serial number of the replaced camera, and determines the motion trajectory corresponding to the replaced camera based on the serial number.

[0117] It should be noted that the time period for changing and the time period for data collection alternate periodically and continuously.

[0118] In some embodiments, the control device can replace the camera mounted on the robotic arm with a camera that has not acquired calibration images in the following ways:

[0119] In some embodiments, the control device controls the robotic arm to replace the camera mounted on the robotic arm with a camera that has not acquired calibration images during a preset replacement time period. This application may also replace the camera on the robotic arm with other devices, which is not limited here.

[0120] In other embodiments, the control device sends a signal to replace the camera during a preset replacement time period, and the operator manually replaces the camera mounted on the robotic arm with a camera that has not acquired calibration images.

[0121] In some embodiments, during camera calibration, the control device can control the robotic arm, camera, and calibration device according to a preset timing control program.

[0122] For example, such as Figure 2As shown, event P0 is the movement of the robotic arm, event P1 is the acquisition of calibration images by the camera, and event P2 is the determination of calibration parameters by the calibration device. The time periods 0 to t1 and t2 to t3 are preset replacement time periods, and the time periods t1 to t2 and t3 to t4 are preset acquisition time periods. The control device controls the movement of the robotic arm and the acquisition of calibration images by the camera during t1 to t2 and t3 to t4. During the time periods 0 to t1 and t2 to t3, the control device replaces the camera on the robotic arm, obtains the serial number of the replaced camera, and determines the corresponding motion trajectory of the replaced camera based on the serial number. The calibration device receives the calibration image sent by the control device at t2 and determines the corresponding calibration parameters of the camera based on the received calibration image. The calibration device also receives the calibration image sent by the control device at t4. During the time period t2 to t5, the calibration device determines the calibration parameters based on the received calibration image, and during the time period t5 to t6, it checks whether the calibration parameters are qualified. Finally, at t6, the calibration device determines the corresponding calibration parameters of the camera based on the received calibration image.

[0123] In this embodiment, after acquiring the calibration image captured by the camera, the control device sends the calibration image to the calibration device. Simultaneously, the calibration device determines the calibration parameters based on the received calibration image, replaces the camera, and controls the replaced camera to acquire the calibration image. Because this embodiment performs the operation of determining the calibration parameters based on the received calibration image in parallel with acquiring the calibration image, i.e., acquiring the calibration image and determining the calibration parameters are processed in parallel, calibration efficiency is improved.

[0124] like Figure 3 As shown in the figure, this application provides a complete flowchart of a camera calibration method applied to a control device, including the following steps:

[0125] Step S301: Obtain the serial number of the camera to be calibrated and determine the motion trajectory corresponding to the serial number;

[0126] Step S302: During the preset acquisition time period, control the robotic arm to drive the camera to move according to the motion trajectory corresponding to the serial number, and control the camera to acquire calibration images by shooting the calibration plate during the movement.

[0127] It should be noted that the camera is mounted on a robotic arm;

[0128] Step S303: Acquire the calibration image captured by the camera and record the calibration image according to the serial number;

[0129] Step S304: During the preset replacement time period, the camera installed on the robotic arm is replaced with a camera that has not acquired calibration images, and the serial number of the replaced camera is obtained. The motion trajectory corresponding to the replaced camera is determined based on the serial number.

[0130] It should be noted that the time period changes and data collection periods alternate periodically and continuously.

[0131] Step S305: Send the calibration image to the calibration device so that the calibration device can determine the calibration parameters corresponding to the camera based on the calibration image;

[0132] It should be noted that steps S304 and S305 are not in any particular order and are executed simultaneously.

[0133] Step S306: Upon receiving a notification from the calibration device that the calibration parameters are not qualified, determine the camera corresponding to the calibration parameters and reclassify the camera into the category of cameras that have not acquired calibration images.

[0134] like Figure 4 As shown, this application embodiment provides a camera calibration system, including a control device 10, a calibration device 40, a robotic arm 20, a camera 30, and a calibration plate 50;

[0135] Control device 10 is used to acquire the serial number of the camera 30 to be calibrated and determine the motion trajectory corresponding to the serial number; during a preset acquisition time period, control the robotic arm 20 to drive the camera 30 to move according to the motion trajectory corresponding to the serial number, and control the camera 30 to acquire calibration images by shooting the calibration plate 50 during the movement; acquire the calibration images acquired by the camera 30 and record the calibration images according to the serial number; send the calibration images to the calibration device 40, and during a preset replacement time period, replace the camera 30 installed on the robotic arm 20 with a camera 30 that has not acquired calibration images, acquire the serial number of the replaced camera 30, and determine the motion trajectory corresponding to the replaced camera 30 according to the serial number; the replacement time period and the acquisition time period alternate periodically and continuously.

[0136] The robotic arm 20 is used to drive the camera 30 to move according to the motion trajectory corresponding to the serial number;

[0137] Camera 30 is used to capture calibration images by photographing calibration plate 50 during movement;

[0138] The calibration device 40 is used to determine the calibration parameters corresponding to the camera 30 based on the calibration image sent by the control device 10. If the calibration parameters are found to be unqualified, the camera 30 corresponding to the calibration parameters is calibrated a second time. If the calibration parameters are found to be qualified, the calibration parameters are stored according to the serial number.

[0139] like Figure 5 As shown in the figure, this application embodiment provides an interactive flowchart of a camera calibration method, which includes the following steps:

[0140] Step S501: The control device installs the camera to be calibrated on the robotic arm, obtains the serial number of the camera to be calibrated, and determines the motion trajectory corresponding to the serial number.

[0141] In step S502, during the preset acquisition time period, the control device controls the robotic arm to drive the camera to move according to the motion trajectory corresponding to the serial number, and controls the camera to acquire calibration images by shooting the calibration plate during the movement.

[0142] In step S503, the camera sends the calibration images acquired during the motion to the control device;

[0143] Step S504: The control device records the received calibration image according to the serial number;

[0144] Step S505: During the preset replacement time period, the control device replaces the camera installed on the robotic arm with a camera that has not acquired calibration images, obtains the serial number of the replaced camera, and determines the motion trajectory corresponding to the replaced camera based on the serial number.

[0145] It should be noted that the time period changes and data collection periods alternate periodically and continuously.

[0146] Step S506: The control device sends the calibration image to the calibration device;

[0147] It should be noted that steps S505 and S506 are not in any particular order and are performed simultaneously.

[0148] Step S507: The calibration device determines the camera's calibration parameters based on the received calibration image;

[0149] Step S508: After the calibration device detects that the determined calibration parameters are qualified, it stores the calibration parameters according to the serial number.

[0150] Based on the same inventive concept, this application also provides a camera calibration device. Since the principle of this device in solving the problem is similar to the camera calibration method in this application, the implementation of this device can refer to the implementation of the method, and the repeated parts will not be described again.

[0151] like Figure 6 As shown, this application embodiment provides a camera calibration device, including:

[0152] The determining unit 601 is used to obtain the serial number of the camera to be calibrated and determine the motion trajectory corresponding to the serial number;

[0153] The control unit 602 is used to control the robotic arm to drive the camera to move according to the motion trajectory corresponding to the serial number during a preset acquisition time period, and to control the camera to acquire calibration images by shooting the calibration plate during the movement; the camera is mounted on the robotic arm;

[0154] The acquisition unit 603 is used to acquire the calibration image captured by the camera and record the calibration image according to the serial number.

[0155] In one optional implementation, after acquiring the calibration image captured by the camera and recording the calibration image according to the serial number, the determining unit 601 is specifically used for:

[0156] During the preset replacement time period, the camera installed on the robotic arm is replaced with a camera that has not acquired calibration images, and the serial number of the replaced camera is obtained. The motion trajectory corresponding to the replaced camera is determined based on the serial number. The replacement time period and the acquisition time period alternate periodically.

[0157] In one optional implementation, after acquiring the calibration image captured by the camera and recording the calibration image according to the serial number, the acquisition unit 603 is further configured to:

[0158] The calibration image is sent to the calibration device so that the calibration device can determine the corresponding calibration parameters for the camera based on the calibration image.

[0159] In one optional implementation, before acquiring the camera's serial number and determining the motion trajectory corresponding to the serial number, the determining unit 601 is further configured to:

[0160] Based on the size of the calibration plate, the field of view of the camera, and the distance between the camera and the calibration plate, determine the position of the robotic arm when the calibration plate appears at the lower left, lower right, upper left, upper right, and center positions of the camera's field of view.

[0161] Based on the determined position of the robotic arm, the range of motion of the robotic arm is determined.

[0162] and Figure 1 The camera calibration method shown is based on the same inventive concept, and embodiments of this application also provide a control device. For example... Figure 7 As shown, for ease of explanation, only the parts related to the embodiments of this application are shown. For specific technical details not disclosed, please refer to the method embodiment section of this application. This electronic device is... Figure 4 The control device 10 shown is described below. In this embodiment, the structure of the control device can be as follows: Figure 7 As shown, it includes a memory 131, a communication module 133, and one or more processors 132.

[0163] The memory 131 is used to store computer programs executed by the processor 132. The memory 131 may mainly include a program storage area and a data storage area. The program storage area may store the operating system and programs required to run instant messaging functions, etc.; the data storage area may store various instant messaging information and operation instruction sets, etc.

[0164] The processor 132 may include one or more central processing units (CPUs) or digital processing units, etc. The processor 132 is used to implement the camera calibration method described above when it calls the computer program stored in the memory 131.

[0165] The communication module 133 is used to communicate with the camera and acquire calibration images.

[0166] This application does not limit the specific connection medium between the memory 131, the communication module 133, and the processor 132 described above. This disclosure embodiment... Figure 7 The memory 131 and the processor 132 are connected via a bus 134, and the bus 134 is in Figure 7 The connections between other components are shown in thick lines only and are not intended to be limiting. The bus 134 can be divided into address bus, data bus, control bus, etc. For ease of illustration, Figure 7 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0167] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the camera calibration method of any of the above embodiments.

[0168] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0169] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0170] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0171] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0172] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A camera calibration method, characterized in that, include: Obtain the first serial number of the first camera to be calibrated, and determine at least one motion trajectory corresponding to the first serial number based on the correspondence between the serial number and the motion trajectory. During a preset acquisition period, the robotic arm is controlled to drive the first camera to move in the order of at least one motion trajectory corresponding to the first serial number, and the first camera is controlled to acquire a first calibration image by shooting a calibration plate during the movement; the first camera is mounted on the robotic arm. The first calibration image is acquired and recorded based on the first serial number, and the first calibration image is sent to the calibration device so that the calibration device determines the first calibration parameters of the first camera based on the received first calibration image. While the calibration device determines the first calibration parameter based on the received first calibration image, the method further includes: Replace the first camera mounted on the robotic arm with a second camera that has not acquired calibration images, and obtain the second serial number of the replaced second camera; Based on the correspondence between the serial number and the motion trajectory, at least one motion trajectory corresponding to the second serial number is determined; The robotic arm is controlled to drive the second camera to move in the order of at least one motion trajectory corresponding to the second serial number, and the second camera is controlled to capture a second calibration image by shooting a calibration plate during the movement; the second calibration image is acquired and recorded based on the second serial number, and the second calibration image is sent to the calibration device so that the calibration device determines the second calibration parameters of the second camera based on the received second calibration image; The replacement time period for controlling the robotic arm to change the corresponding camera and the acquisition time period for controlling the robotic arm to drive the corresponding camera to acquire calibration images alternate periodically and continuously.

2. The method according to claim 1, characterized in that, Before obtaining the first serial number of the first camera and determining at least one motion trajectory corresponding to the first serial number, the method further includes: Based on the size of the calibration plate, the field of view of the camera, and the distance between the camera and the calibration plate, the position points of the robotic arm are determined when the calibration plate appears at the lower left, lower right, upper left, upper right, and center positions of the camera's field of view, respectively. The range of motion of the robotic arm is determined based on its position.

3. A camera calibration device, characterized in that, include: The determining unit is used to obtain the first serial number of the first camera to be calibrated, and determine at least one motion trajectory corresponding to the first serial number based on the correspondence between the serial number and the motion trajectory. The control unit is used to control the robotic arm to drive the first camera to move in the order of at least one motion trajectory corresponding to the first serial number during a preset acquisition time period, and to control the first camera to acquire a first calibration image by shooting a calibration plate during the movement; the first camera is mounted on the robotic arm. The acquisition unit is configured to acquire and record the first calibration image based on the first serial number, and send the first calibration image to the calibration device so that the calibration device determines the first calibration parameters of the first camera based on the received first calibration image. The determining unit is further configured to, while the calibration device determines the first calibration parameters based on the received first calibration image, replace the first camera mounted on the robotic arm with a second camera that has not acquired calibration images, and obtain the second serial number of the replaced second camera; and determine at least one motion trajectory corresponding to the second serial number based on the correspondence between the serial number and the motion trajectory. The control unit is further configured to control the robotic arm to drive the second camera to move in the order of at least one motion trajectory corresponding to the second serial number, and to control the second camera to acquire a second calibration image by shooting a calibration plate during the movement; to acquire and record the second calibration image based on the second serial number, and to send the second calibration image to the calibration device so that the calibration device can determine the second calibration parameters of the second camera based on the received second calibration image; The replacement time period for controlling the robotic arm to change the corresponding camera and the acquisition time period for controlling the robotic arm to drive the corresponding camera to acquire calibration images alternate periodically and continuously.

4. A control device, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program that can run on the processor, and when the computer program is executed by the processor, causes the processor to perform the method of claim 1 or 2.

5. A camera calibration system, characterized in that, Includes control equipment, calibration equipment, robotic arms, and cameras; The control device is configured to acquire the first serial number of the first camera to be calibrated, determine at least one motion trajectory corresponding to the first serial number based on the correspondence between the serial number and the motion trajectory, control the robotic arm to drive the first camera to move in the order of at least one motion trajectory corresponding to the first serial number during a preset acquisition time period, and control the first camera to acquire a first calibration image by shooting a calibration plate during the movement; acquire and record the first calibration image based on the first serial number, and send the first calibration image to the calibration device. While the calibration device determines the first calibration parameters based on the received first calibration image, the first camera mounted on the robotic arm is replaced with a second camera that has not acquired calibration images, and the second serial number of the replaced second camera is obtained; based on the correspondence between the serial number and the motion trajectory, at least one motion trajectory corresponding to the second serial number is determined; the robotic arm is controlled to drive the second camera to move in the order of at least one motion trajectory corresponding to the second serial number, and the second camera is controlled to acquire a second calibration image by shooting the calibration board during the movement; the second calibration image is acquired and recorded based on the second serial number, and the second calibration image is sent to the calibration device; The robotic arm is used to drive the first camera to move in the order of at least one motion trajectory corresponding to the first serial number, and to drive the second camera to move in the order of at least one motion trajectory corresponding to the second serial number. The camera is used to capture calibration images of the calibration plate during movement; The calibration device is used to determine the calibration parameters of a corresponding camera based on the calibration image sent by the control device, the calibration image including the first calibration image and the second calibration image; if the calibration parameters are detected to be unqualified, the camera corresponding to the calibration parameters is calibrated a second time. If the calibration parameters are found to be qualified, the calibration parameters are stored according to the corresponding serial number; The replacement time period for controlling the robotic arm to change the corresponding camera and the acquisition time period for controlling the robotic arm to drive the corresponding camera to acquire calibration images alternate periodically and continuously.

6. A computer storage medium storing a computer program, characterized in that: When the instructions are executed on a computer, the computer performs the method of claim 1 or 2.

Images