Camera extrinsic parameter calibration device, calibration method, electronic equipment and medium

The camera extrinsic parameter calibration device, which combines a U-shaped guide rail and a laser rangefinder, solves the problems of cumbersome calibration process and limited accuracy, achieving efficient and accurate camera extrinsic parameter calibration, reducing errors, and improving data processing efficiency.

CN114708329BActive Publication Date: 2024-11-29ZHONGCHUAN TIANXIA INFORMATION TECH BEIJING CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202210157665.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-02-21
Publication Date
2024-11-29
Estimated Expiration
2042-02-21

AI Technical Summary

Technical Problem

In existing technologies, the calibration process for camera extrinsic parameters is cumbersome and has limited accuracy. Especially in multi-camera systems, errors are easily generated during hardware placement and use, resulting in large data processing errors.

Method used

A U-shaped guide rail and a laser rangefinder are combined with a camera extrinsic parameter calibration device. The camera moves and the lens rotates on the U-shaped guide rail through the control unit. The positional relationship between the camera, the laser rangefinder, and the camera is determined by the laser rangefinder and a rotary encoder. The extrinsic parameters are calculated using the Rodriguez formula.

Benefits of technology

It reduces the workload of manual calibration, avoids the cumulative effect of subtle changes in hardware placement and use that lead to external parameter errors, and improves data accuracy and processing efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114708329B_ABST
    Figure CN114708329B_ABST
Patent Text Reader

Abstract

The present application discloses a camera extrinsic calibration device, calibration method, electronic device and medium, the device comprises: a U-shaped guide rail, a laser rangefinder and at least one camera, the U-shaped guide rail has a slot along the length direction, the laser rangefinder is arranged on the bottom surface of one end of the U-shaped guide rail along the length direction, and is used to measure the distance between the laser rangefinder and at least one camera, and the at least one camera comprises a base, a control unit, a displacement unit, a yaw axis, a pitch axis, a lens, a semi-transparent and semi-reflective plate and a rotary encoder; the method comprises: determining the extrinsic parameter relationship between at least one camera and the laser rangefinder according to the acquired distance between at least one camera and the laser rangefinder, the angle of rotation of at least one camera lens in the horizontal direction and the angle of rotation of at least one camera lens in the vertical direction. The present application obtains displacement and angle parameters in real time, has higher data accuracy, improves the efficiency of later data processing, and greatly reduces the workload of calibration personnel.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of camera calibration, and in particular to a camera extrinsic calibration device, a calibration method, an electronic device and a medium. Background Art

[0002] At a time when autonomous driving-related businesses are booming, high-precision maps and road information, as data support, have played an indelible role in the autonomous driving industry, and on-board data collection equipment has played a huge role in the high-precision map and road information collection business.

[0003] For vehicle-mounted acquisition equipment, the external parameter calibration between cameras has always been a cumbersome process. For example, in the prototype design stage, the baseline and orientation between cameras will be frequently updated, and each change to the camera requires an external parameter calibration. For example, in the field operation process, for the safety of the acquisition equipment, the equipment needs to be disassembled after each acquisition task. Frequent loading and unloading will cause errors in the actual external parameters and the external parameters of the initial calibration, which will cause problems in subsequent data processing. In addition, when there are a large number of cameras, the work of calibrating the external parameters of each camera is repetitive and arduous.

[0004] Therefore, in the prior art, there is a lack of a device and method that can efficiently and accurately calibrate the external parameters of a multi-eye camera without increasing the calibration workload. Summary of the invention

[0005] To solve the above problems, the embodiments of the present application provide a camera extrinsic parameter calibration device, a calibration method, an electronic device and a medium, which aim to solve the problem of rapid calibration of extrinsic parameters between a camera and a laser rangefinder, and between cameras.

[0006] The present application embodiment adopts the following technical solutions:

[0007] In a first aspect, a camera extrinsic calibration device is provided, the device comprising: a U-shaped guide rail, a laser rangefinder and at least one camera; wherein:

[0008] The U-shaped guide rail has slots along the length direction;

[0009] The laser rangefinder is arranged on the bottom surface of one end of the U-shaped guide rail along the length direction, and is used to measure the distance between the laser rangefinder and at least one camera;

[0010] One of the cameras includes: a base, a control unit, a displacement unit, a yaw axis, a pitch axis, a lens, a semi-transparent and semi-reflective plate, and a rotary encoder; wherein,

[0011] The base is inserted into the slot of the U-shaped guide rail;

[0012] The control unit is electrically connected to the displacement unit, the yaw axis, and the pitch axis, respectively, and is used to control at least one of the displacement unit, the yaw axis, and the pitch axis;

[0013] The displacement unit is used to move the base along the length direction of the U-shaped guide rail according to the control of the control unit;

[0014] The yaw shaft is connected to the base and the lens respectively, and is used to rotate the lens in the horizontal direction according to the control of the control unit;

[0015] The pitch axis is connected to the base and the lens respectively, and is used to rotate the lens in the vertical direction according to the control of the control unit;

[0016] The semi-transparent and semi-reflective plate is arranged at the lower part of the base, and is used to receive and reflect the laser emitted by the laser rangefinder;

[0017] The rotary encoder is connected to the yaw axis and the pitch axis, and is used to collect the angle of rotation of the lens in the horizontal direction and the angle of rotation of the lens in the vertical direction.

[0018] Optionally, the control unit includes: a plurality of stepper motors; wherein,

[0019] The plurality of stepper motors are respectively used to drive the displacement unit to move a corresponding distance, drive the yaw axis to rotate a corresponding angle, and drive the pitch axis to rotate a corresponding angle according to the control signal.

[0020] Optionally, the device further includes: a wireless transceiver unit and a terminal device; wherein,

[0021] The wireless transceiver unit is electrically connected to the control unit, and the wireless transceiver unit is wirelessly connected to the terminal device;

[0022] The wireless transceiver unit is used to receive the control signal output by the terminal device and transmit it to the control unit.

[0023] Optionally, the device further comprises: a display unit; wherein,

[0024] The display unit is connected to the laser rangefinder and the rotary encoder respectively, and is used to display at least one of the distance between the laser rangefinder and the camera, the angle of rotation of the lens in the horizontal direction, and the angle of rotation of the lens in the vertical direction.

[0025] Optionally, the displacement unit comprises: a gear belt; wherein,

[0026] The gear belt is arranged on the base, and the gears of the gear belt face and contact the U-shaped guide rail;

[0027] The gear belt drives the base to move along the length direction of the U-shaped guide rail according to the control of the control unit.

[0028] Optionally, the central axis of the semi-transparent and semi-reflective plate is perpendicular to the optical axis of the lens, and the plane formed by the central axis of the semi-transparent and semi-reflective plate and the optical axis of the lens evenly divides the semi-transparent and semi-reflective plate into two parts.

[0029] In a second aspect, a camera extrinsic calibration method is provided, which is implemented using the above-mentioned camera extrinsic calibration device, and the method includes:

[0030] Acquire the distance between at least one camera and the laser rangefinder, the horizontal rotation angle of at least one camera lens, and the vertical rotation angle of at least one camera lens;

[0031] An external parameter relationship between the at least one camera and the laser rangefinder is determined according to the distance between the at least one camera and the laser rangefinder, the angle of rotation of the at least one camera lens in the horizontal direction, and the angle of rotation of the at least one camera lens in the vertical direction.

[0032] Optionally, when there are multiple cameras, the method further includes:

[0033] According to the external parameter relationship between multiple cameras and the laser rangefinder, the angular position relationship between any two cameras is determined.

[0034] In a third aspect, an embodiment of the present application further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned camera extrinsic parameter calibration method when executing the computer program.

[0035] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores a computer program, and the computer program implements the steps of the above-mentioned camera extrinsic parameter calibration method when instructed by a processor.

[0036] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects:

[0037] The camera extrinsic parameter calibration device of the present application inserts the camera into a U-shaped guide rail, controls the movement of the camera along the length direction of the U-shaped guide rail, and controls the rotation of the camera lens in the horizontal and vertical directions of freedom through a control unit. It only needs to select a laser rangefinder as a reference to conveniently determine the relative position of the camera and the laser rangefinder; and then, when there are multiple cameras, determine the relative position of each camera. In addition, the number of cameras can be flexibly increased or decreased through the slots of the U-shaped guide rail. The camera extrinsic parameter calibration device and calibration method of the present application greatly reduce the workload of manual calibration, and avoid the extrinsic parameter errors caused by the cumulative effect of subtle changes in hardware placement and use. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0039] Figure 1 A schematic diagram showing the structure of a camera extrinsic calibration device according to an embodiment of the present application is shown;

[0040] Figure 2 A schematic diagram showing the structure of a U-shaped guide rail according to an embodiment of the present application is shown;

[0041] Figure 3 A schematic structural diagram of a camera according to an embodiment of the present application is shown;

[0042] Figure 4 A schematic diagram showing a flow chart of a camera extrinsic calibration method according to an embodiment of the present application is shown;

[0043] Figure 5 A schematic structural diagram of an electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0045] First, the existing camera calibration technology is introduced.

[0046] Camera calibration refers to the calibration of the camera's intrinsic and extrinsic parameters. Since the camera's projection model needs to be associated with the scale of the real world, calibration is necessary. Traditional calibration methods generally use objects of known size in the real world to solve internal and external parameters through mathematical and physical models. For example, Zhang Dingyou's calibration method uses a chessboard of known size and sets the upper left corner as the origin of the world coordinate system to calculate the internal and external parameters.

[0047] During the camera calibration process, if you need to rely on objects of fixed size, the calibration process will be cumbersome. Therefore, the current mainstream calibration method is self-calibration, which is to approximate the internal and external parameters of the camera by constraining the camera's own parameters, applying geometric constraints to the obtained image, and optimizing the error terms.

[0048] Regardless of the calibration method, calibration requires knowing some physical hardware parameters, such as the baseline distance of the binocular camera, the axis of the camera optical axis, etc. Current autonomous driving collection vehicles are generally equipped with multiple cameras, and the relative position and direction relationship between each camera needs to be transformed as accurately as possible, because these relationships are used as system constants, and their accuracy directly affects the error size of subsequent data processing. The difficulty of multi-camera calibration is that the position relationship between the laser rangefinder and the camera, and between different cameras is usually obtained through mechanical measurement, and the deviation that may exist during the placement process is superimposed, resulting in a large error in camera calibration.

[0049] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0050] The idea of ​​this application is to provide a camera extrinsic parameter calibration device and method to address the shortcomings of existing camera calibration technology, such as complicated process and accuracy limited by physical hardware and spatial structure, so as to greatly reduce the workload of manual calibration and avoid extrinsic parameter errors caused by the cumulative effect of subtle changes in hardware placement and use.

[0051] The camera extrinsic parameter calibration device of the present application includes: a U-shaped guide rail 1, a laser rangefinder 2 and at least one camera 3.

[0052] The U-shaped guide rail 1 has a slot 101 along the length direction. The laser rangefinder 2 is arranged on the bottom surface of one end of the U-shaped guide rail 1 along the length direction, and is used to measure the distance between the laser rangefinder 2 and at least one camera 3. The at least one camera 3 includes: a base 301, a control unit 302, a displacement unit 303, a yaw axis 304, a pitch axis 305, a lens 306, a semi-transparent and semi-reflective plate 307 and a rotary encoder 308. The base 301 is inserted into the slot 101 of the U-shaped guide rail 1; the control unit 302 is electrically connected to the displacement unit 303, the yaw axis 304, and the pitch axis 305, respectively, and is used to control at least one of the displacement unit 303, the yaw axis 304 and the pitch axis 305; the displacement unit 303 is used to move the base 301 along the length direction of the U-shaped guide rail 1 according to the control of the control unit 302; the yaw axis 304 is connected to the base 301 and the lens 306, respectively, and is used to control the base 301 according to the control of the control unit 302. The yaw axis 304 and the pitch axis 305 are connected to the base 301 and the lens 306 respectively, and are used to rotate the lens 306 in the vertical direction according to the control of the control unit 302; the semi-transparent and semi-reflective plate 307 is arranged at the lower part of the base 301, and is used to receive and reflect the laser emitted by the laser rangefinder 2; the rotary encoder 308 is connected to the yaw axis 304 and the pitch axis 305, and is used to collect the angle of rotation of the lens 306 in the horizontal direction and the angle of rotation of the lens 306 in the vertical direction.

[0053] Figure 1 A schematic diagram of the structure of a camera extrinsic calibration device according to an embodiment of the present application is shown. Figure 2 A schematic diagram showing the structure of a U-shaped guide rail according to an embodiment of the present application is shown. Figure 3 FIG. 1 shows a schematic diagram of a camera structure according to an embodiment of the present application. Figures 1 to 3 The example introduces the camera extrinsic calibration device in detail.

[0054] Combination Figure 2 As shown, concave slots 101 are symmetrically provided on the outer sides of the two wings of the U-shaped guide rail 1 along the length direction. Figure 3 As shown, the two wings of the base 301 of the camera 3 have symmetrical bosses, and the bosses of the base 301 of the camera 3 match the concave slots 101 of the U-shaped guide rail 1, so that the camera 3 can be inserted into the U-shaped guide rail 1. Figure 2 and Figure 3 The matching mode of the U-shaped guide rail 1 and the base 301 of the camera 3 shown is only an illustrative implementation mode. In actual application, the matching mode of the slot 101 of the U-shaped guide rail 1 and the base 301 of the camera 3 is not limited to this. For example: the upper sides of the two wings of the U-shaped guide rail 1 are symmetrically provided with concave slots 101 along the length direction, and the lower part of the base 301 of the camera 3 has a symmetrical boss that can match the concave slot 101, so that the camera 3 can be inserted into the U-shaped guide rail 1; for another example: the upper side of one wing of the U-shaped guide rail 1 is provided with a concave slot 101 along the length direction, and the lower part of the base 301 of the camera 3 has a boss that can match the concave slot 101, so that the camera 3 can be inserted into the U-shaped guide rail 1. In other words, as long as the base 301 of the camera 3 can be inserted into the slot 101 opened along the length direction of the U-shaped guide rail 1, it will be fine.

[0055] Combination Figure 2 As shown, the laser rangefinder 2 is arranged on the bottom surface of one end of the U-shaped guide rail 1 along the length direction, and emits laser light along the length direction of the U-shaped guide rail 1 toward the other end. Figure 3 As shown, a semi-transparent and semi-reflective plate 307 is provided at the lower part of the base 301 of the camera 3. When the laser emitted by the laser rangefinder 2 reaches the semi-transparent and semi-reflective plate 307, it is received and reflected by the semi-transparent and semi-reflective plate 307, and the laser rangefinder 2 receives the reflected laser to measure the distance between the laser rangefinder 2 and the camera 3. In particular, when more than one camera 3 is required, the camera 3 can be flexibly added from the end of the U-shaped guide rail 1 away from the laser rangefinder 2. The semi-transparent and semi-reflective plates 307 of each camera 3 can select different reflection intensities, so that each camera 3 can be distinguished according to the intensity of each reflected light received by the laser rangefinder 2.

[0056] Combination Figure 3As shown, the camera 3 includes: a base 301, a control unit 302, a displacement unit 303, a yaw axis 304, a pitch axis 305, a lens 306, a semi-transparent and semi-reflective plate 307, and a rotary encoder 308. The base 301 and the semi-transparent and semi-reflective plate 307 have been described in detail before and will not be repeated here. The control unit 302, the displacement unit 303, the yaw axis 304, the pitch axis 305, the lens 306, and the rotary encoder 308 of the camera 3 are explained in detail below.

[0057] The control unit 302 is electrically connected to the displacement unit 303, the yaw axis 304, and the pitch axis 305, respectively, and is used to control at least one of the displacement unit 303, the yaw axis 304, and the pitch axis 305. The control unit 302 can be disposed on the base 301 of the camera 3, and controls the displacement unit 303 so that the base 301 drives the lens 306 to move, and / or controls the yaw axis 304 and / or the pitch axis 305 to drive the lens 306 to rotate.

[0058] The control unit 302 may be an integrated unit module, which can selectively control at least one of the displacement unit 303, the yaw axis 304, and the pitch axis 305 according to the needs of the user. For example, when the camera 3 lens 306 only needs to move without rotating, the control unit 302 only controls the displacement unit 303 according to the user's selection. The control unit 302 may also include a plurality of independent control modules, each of which controls the displacement unit 303, the yaw axis 304, and the pitch axis 305 respectively. For example, when the camera 3 lens 306 only needs to move without rotating, the independent control module corresponding to the displacement unit 303 controls the displacement unit 303 according to the user's selection.

[0059] The user can manually or automatically control the displacement unit 303, the yaw axis 304, and the pitch axis 305 through the control unit 302 as needed. When the user needs to manually control the camera 3, the control unit 302 in the form of a mechanical or electronic switch, such as a photoelectric switch, can be used; when the user needs to automatically control the camera 3, the control unit 302 in the form of a programming form can be used to continuously or regularly control the displacement unit 303, the yaw axis 304, and the pitch axis 305.

[0060] The displacement unit 303 moves the base 301 along the length direction of the U-shaped guide rail 1 according to the control of the control unit 302 , thereby driving the lens 306 to move along the length direction of the U-shaped guide rail 1 .

[0061] The yaw shaft 304 is connected to the base 301 and the lens 306 respectively, and according to the control of the control unit 302, the lens 306 is rotated in the horizontal direction. Figure 3Taking the marking shown in the right figure as an example, the lens 306 is driven by the yaw shaft 304 to rotate horizontally clockwise or counterclockwise with the vertical direction as the axis.

[0062] The pitch axis 305 is connected to the base 301 and the lens 306 respectively, and according to the control of the control unit 302, the lens 306 is rotated in the vertical direction. Figure 3 Taking the marking shown in the left figure as an example, the lens 306 is driven by the pitch axis 305 to rotate vertically clockwise or counterclockwise with the horizontal direction as the axis.

[0063] The rotary encoder 308 is connected to the yaw axis 304 and the pitch axis 305, and the rotary encoder 308 collects the angle of rotation of the lens 306 in the horizontal direction and the angle of rotation of the lens 306 in the vertical direction. The rotary encoder 308 can be used to convert mechanical quantities such as the angular displacement or angular velocity of the yaw axis 304 and the pitch axis 305 into corresponding electrical pulses and output them as digital quantities, thereby collecting the angle of rotation of the lens 306 in the horizontal direction and the angle of rotation of the lens 306 in the vertical direction.

[0064] In the present application, the horizontal direction and the vertical direction are directions based on the earth coordinate system.

[0065] As an optional implementation, the control unit 302 includes: a plurality of stepper motors, which are respectively used to drive the displacement unit 303 to move a corresponding distance, drive the yaw axis 304 to rotate a corresponding angle, and drive the pitch axis 305 to rotate a corresponding angle according to the control signal.

[0066] A stepper motor is an electric motor that converts an electrical pulse signal into a corresponding angular displacement or linear displacement. Each time a pulse signal is input, the rotor rotates an angle or moves forward one step. The control unit 302 includes a plurality of stepper motors that are electrically connected to the displacement unit 303, the yaw axis 304, and the pitch axis 305, respectively. The user controls the camera 3 lens 306 through a plurality of stepper motors according to his or her own needs. When the user needs to move the camera 3 lens 306 along the U-shaped guide rail 1 by a corresponding distance, the linear acceleration can be adjusted by the stepper motor of the control unit 302, the corresponding linear displacement can be calculated by integration, and the displacement unit 303 can be driven to move by the stepper motor, so that the camera 3 lens 306 moves along the U-shaped guide rail 1 by a corresponding distance. When the user needs to rotate the camera 3 lens 306 in the horizontal direction by a corresponding angle, the angular acceleration can be adjusted by the stepper motor of the control unit 302, the corresponding angular displacement can be calculated by integration, and the yaw axis 304 can be driven to rotate by the stepper motor, so that the camera 3 lens 306 rotates in the horizontal direction. When the user needs to rotate the camera 3 lens 306 in the vertical direction to a corresponding angle, the user can adjust the angular acceleration through the stepper motor of the control unit 302, calculate the corresponding angular displacement through integration, and use the stepper motor to drive the pitch axis 305 to rotate, so that the camera 3 lens 306 rotates in the vertical direction.

[0067] As an optional implementation, the camera external parameter calibration device further includes: a wireless transceiver unit and a terminal device. The wireless transceiver unit is electrically connected to the control unit 302, and the wireless transceiver unit is wirelessly connected to the terminal device. The wireless transceiver unit receives the control signal output by the terminal device and transmits it to the control unit 302.

[0068] The wireless connection between the wireless transceiver unit and the terminal device may be, but not limited to, infrared, Bluetooth, or wireless connection. The terminal device may be, but not limited to, a remote controller, a smart device, a server, or other terminal capable of transmitting control signals. When the terminal device is in the form of a smart device or a server, the control signal may be output through software installed or embedded therein.

[0069] As an optional implementation, the camera extrinsic calibration device further includes: a display unit. The display unit is connected to the laser rangefinder 2 and the rotary encoder 308, respectively, and is used to display at least one of the distance between the laser rangefinder 2 and the camera 3, the angle of rotation of the lens 306 in the horizontal direction, and the angle of rotation of the lens 306 in the vertical direction.

[0070] The laser rangefinder 2 emits a laser, which is reflected by the semi-transparent and semi-reflective plate 307 of the camera 3 and then transmitted back to the receiver of the laser rangefinder 2. The distance between the camera 3 and the laser rangefinder 2 can be determined by the femtosecond method or the triangulation method. Both the femtosecond method and the triangulation method are relatively mature laser distance measurement methods in the field, and will not be described in detail here. The display unit is connected to the laser rangefinder 2 and the rotary encoder 308, and at least one of the three data of the distance between the laser rangefinder 2 and the camera 3, the angle of rotation of the lens 306 in the horizontal direction collected by the rotary encoder 308, and the angle of rotation of the lens 306 in the vertical direction is displayed.

[0071] As an optional embodiment, the displacement unit 303 includes: a gear belt. The gear belt is arranged on the base 301, and the gears of the gear belt face and contact the U-shaped guide rail 1. The gear belt drives the base 301 to move along the length direction of the U-shaped guide rail 1 according to the control of the control unit 302.

[0072] In order to increase the controllability of the camera 3 base 301 during movement, the displacement unit 303 also includes a gear belt. Figure 3 In the direction shown in the left figure, the gear belt is hidden on the base 301, and the gears of the gear belt face and contact the U-shaped guide rail 1. In one case, the gear belt can be set to be ring-shaped, and the control unit 302 can include a stepper motor. In this case, when the user needs to move the camera 3 lens 306 along the U-shaped guide rail 1 for a corresponding distance, the angular acceleration can be adjusted by the stepper motor of the control unit 302, and the corresponding angular displacement can be calculated by integration, and the stepper motor can be used to drive the ring-shaped gear belt to rotate, thereby driving the camera 3 lens 306 to move the corresponding distance along the U-shaped guide rail 1.

[0073] As an optional implementation, the center axis of the semi-transparent and semi-reflective plate 307 is perpendicular to the optical axis of the lens 306 , and the plane formed by the center axis of the semi-transparent and semi-reflective plate 307 and the optical axis of the lens 306 evenly divides the semi-transparent and semi-reflective plate 307 into two parts.

[0074] In order to ensure accurate distance measurement, the central axis of the semi-transparent and semi-reflective plate 307 is perpendicular to the optical axis of the lens 306. The plane formed by the central axis of the semi-transparent and semi-reflective plate 307 and the optical axis of the lens 306 evenly divides the semi-transparent and semi-reflective plate 307 into two parts, thereby ensuring that no systematic error will occur when the lens 306 moves and rotates.

[0075] When the camera extrinsic calibration device is used to calibrate multiple cameras 3, the distance between the laser rangefinder 2 and each camera 3 is determined by the laser rangefinder 2, and the axis angle of rotation of the lens 306 of each camera 3 is determined by the rotary encoder 308. The axis angle of rotation of the lens 306 of each camera 3 is matrix transformed according to the Rodriguez formula, so that the extrinsic parameter relationship between each camera 3 and the laser rangefinder 2 can be determined, and then the angular position relationship between each camera 3 can be determined very conveniently with the laser rangefinder 2 as a reference.

[0076] Figure 4 A schematic flow chart of a camera extrinsic calibration method according to an embodiment of the present application is shown. The method is implemented using the above-mentioned camera extrinsic calibration device.

[0077] from Figure 4 It can be seen that the present application at least includes steps S410 to S420:

[0078] Step S410: obtaining the distance between at least one camera and the laser rangefinder, the horizontal rotation angle of at least one camera lens, and the vertical rotation angle of at least one camera lens.

[0079] The distance between at least one camera and the laser rangefinder is acquired in real time by the laser rangefinder. The laser rangefinder emits laser light to at least one camera, and the semi-transparent and semi-reflective plate of at least one camera receives and reflects the laser light back to the receiver of the laser rangefinder. The laser rangefinder determines the distance between at least one camera and the laser rangefinder according to the femtosecond method or the triangulation method. When there are multiple cameras, the reflection intensity of the semi-transparent and semi-reflective plates of each camera is different to distinguish different cameras.

[0080] The horizontal rotation angle of at least one camera lens and the vertical rotation angle of at least one camera lens are acquired in real time through a rotary encoder of at least one camera. The rotary encoder is used to convert mechanical quantities such as angular displacement or angular velocity of the yaw axis and the pitch axis into corresponding electrical pulses and output them as digital quantities, thereby acquiring the horizontal rotation angle of the lens and the vertical rotation angle of the lens.

[0081] Step S420: determining an external parameter relationship between at least one camera and the laser rangefinder according to the distance between the at least one camera and the laser rangefinder, the horizontal rotation angle of the at least one camera lens, and the vertical rotation angle of the at least one camera lens.

[0082] The horizontal rotation angle of the camera lens and the vertical rotation angle of the camera together constitute the components of the camera lens rotation angle vector in each axis. After determining the axis angle of the camera lens rotation, the matrix representation of the camera lens rotation can be obtained according to the Rodriguez formula. Combined with the distance between the camera and the laser rangefinder, the external parameter relationship between the camera and the laser rangefinder is determined.

[0083] The Rodriguez formula is expressed as follows:

[0084] R=I+sin(θ)K+(1-cos(θ))K 2 .

[0085] Among them, R represents the rotation matrix, I represents the identity matrix, θ represents the rotation angle, and K represents the "cross product matrix" of the unit vector k of the rotation axis.

[0086] As an optional implementation, when there are multiple cameras, the camera extrinsic parameter calibration method further includes: determining the angular position relationship between any two cameras according to the extrinsic parameter relationship between the multiple cameras and the laser rangefinder.

[0087] When there are multiple cameras, the external parameter relationship between each camera and the laser rangefinder can be determined. Therefore, taking the laser rangefinder as a reference, the external parameter relationship between any two cameras and the laser rangefinder is superimposed to determine the angular position relationship between any two cameras.

[0088] The camera extrinsic calibration device and calibration method of the present application obtains the distance between each camera and the laser rangefinder in real time through a laser rangefinder, and obtains the horizontal rotation angle and vertical rotation angle of the camera lens in real time through a rotary encoder, thereby determining the extrinsic parameter relationship between each camera and the laser rangefinder through the Rodriguez formula, and then determining the angular position relationship between each camera based on the laser rangefinder. The technical solution of the present application can reduce the workload of calibration personnel, avoid the internal and external parameter errors caused by the accumulation effect of subtle changes in the placement and use of the camera, and improve the data accuracy and the efficiency of later data processing.

[0089] Figure 5 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 5 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0090] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0091] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0092] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it. The processor executes the program stored in the memory and is specifically used to perform the following operations:

[0093] Acquire the distance between at least one camera and the laser rangefinder, the horizontal rotation angle of at least one camera lens, and the vertical rotation angle of at least one camera lens;

[0094] An external parameter relationship between the at least one camera and the laser rangefinder is determined according to the distance between the at least one camera and the laser rangefinder, the angle of rotation of the at least one camera lens in the horizontal direction, and the angle of rotation of the at least one camera lens in the vertical direction.

[0095] The above application Figure 4The camera extrinsic calibration method disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or instructions in the form of software. The above processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in a decoding processor. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0096] The electronic device may also perform Figure 4 The camera extrinsic calibration method will not be described in detail in the embodiments of the present application.

[0097] The present application also provides a computer-readable storage medium, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by an electronic device including multiple application programs, enable the electronic device to execute Figure 4 The camera extrinsic calibration method in the illustrated embodiment is specifically used to perform:

[0098] Acquire the distance between at least one camera and the laser rangefinder, the horizontal rotation angle of at least one camera lens, and the vertical rotation angle of at least one camera lens;

[0099] An external parameter relationship between the at least one camera and the laser rangefinder is determined according to the distance between the at least one camera and the laser rangefinder, the angle of rotation of the at least one camera lens in the horizontal direction, and the angle of rotation of the at least one camera lens in the vertical direction.

[0100] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0101] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0102] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0103] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0104] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0105] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0106] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0107] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0108] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0109] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A camera extrinsic calibration device, characterized in that: The device comprises: a U-shaped guide rail, a laser rangefinder and at least one camera; wherein, The U-shaped guide rail has slots along the length direction; The laser rangefinder is arranged on the bottom surface of one end of the U-shaped guide rail along the length direction, and is used to measure the distance between the laser rangefinder and at least one of the cameras; At least one of the cameras comprises: a base, a control unit, a displacement unit, a yaw axis, a pitch axis, a lens, a semi-transparent and semi-reflective plate, and a rotary encoder; wherein, The base is inserted into the slot of the U-shaped guide rail; The control unit is electrically connected to the displacement unit, the yaw axis, and the pitch axis, respectively, and is used to control at least one of the displacement unit, the yaw axis, and the pitch axis; The displacement unit is used to move the base along the length direction of the U-shaped guide rail according to the control of the control unit; The yaw shaft is connected to the base and the lens respectively, and is used to rotate the lens in the horizontal direction according to the control of the control unit; The pitch axis is connected to the base and the lens respectively, and is used to rotate the lens in the vertical direction according to the control of the control unit; The semi-transparent and semi-reflective plate is arranged at the lower part of the base, and is used to receive and reflect the laser emitted by the laser rangefinder; The rotary encoder is connected to the yaw axis and the pitch axis, and is used to collect the angle of rotation of the lens in the horizontal direction and the angle of rotation of the lens in the vertical direction.

2. The camera extrinsic calibration device according to claim 1, characterized in that: The control unit comprises: a plurality of stepper motors; wherein, The plurality of stepper motors are respectively used to drive the displacement unit to move a corresponding distance, drive the yaw axis to rotate a corresponding angle, and drive the pitch axis to rotate a corresponding angle according to a control signal.

3. The camera extrinsic calibration device according to claim 1, characterized in that: The device also includes: a wireless transceiver unit and a terminal device; wherein, The wireless transceiver unit is electrically connected to the control unit, and the wireless transceiver unit is wirelessly connected to the terminal device; The wireless transceiver unit is used to receive the control signal output by the terminal device and transmit it to the control unit.

4. The camera extrinsic calibration device according to claim 1, characterized in that: The device further comprises: a display unit; wherein, The display unit is connected to the laser rangefinder and the rotary encoder respectively, and is used to display at least one of the distance between the laser rangefinder and the camera, the angle of rotation of the lens in the horizontal direction, and the angle of rotation of the lens in the vertical direction.

5. The camera extrinsic calibration device according to claim 1, characterized in that: The displacement unit comprises: a gear belt; wherein, The gear belt is arranged on the base, and the gears of the gear belt face and contact the U-shaped guide rail; The gear belt drives the base to move along the length direction of the U-shaped guide rail according to the control of the control unit.

6. The camera extrinsic calibration device according to claim 1, characterized in that: The central axis of the semi-transparent and semi-reflective plate is perpendicular to the optical axis of the lens, and the plane formed by the central axis of the semi-transparent and semi-reflective plate and the optical axis of the lens evenly divides the semi-transparent and semi-reflective plate into two parts.

7. A camera extrinsic calibration method, characterized in that: The method is implemented by using the camera extrinsic calibration device according to any one of claims 1 to 6, and comprises: Acquire the distance between at least one camera and the laser rangefinder, the horizontal rotation angle of at least one camera lens, and the vertical rotation angle of at least one camera lens; An external parameter relationship between the at least one camera and the laser rangefinder is determined according to the distance between the at least one camera and the laser rangefinder, the angle of rotation of the at least one camera lens in the horizontal direction, and the angle of rotation of the at least one camera lens in the vertical direction.

8. The camera extrinsic calibration method according to claim 7, characterized in that: When there are multiple cameras, the method further includes: According to the external parameter relationship between multiple cameras and the laser rangefinder, the angular position relationship between any two cameras is determined.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the camera extrinsic parameter calibration method as described in any one of claims 7-8 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is instructed by the processor, it implements the steps of the camera extrinsic parameter calibration method as described in any one of claims 7-8.

Citation Information

Patent Citations

  • Infrared control method and system supporting multiple coding formats

    CN110930684A

  • Camera calibration method and device for structured light depth measurement

    CN111161358A