A camera extrinsic parameter calibration method and system based on line laser equipment

Abstract

The present invention relates to the field of road engineering, and specifically discloses a camera extrinsic parameter calibration method and system based on a line laser device, the method comprising setting parameters for a vehicle-mounted device; the vehicle-mounted device comprises a vehicle-mounted camera and a line laser device; sending a transmission instruction to the line laser device according to a preset first frequency, and sending a collection instruction to the vehicle-mounted camera according to a preset second frequency; wherein the first frequency is higher than the second frequency; receiving an image collected by the vehicle-mounted camera, identifying the image, and determining pixel coordinates; and calibrating the camera extrinsic parameters according to the pixel coordinates. The present invention is simple to operate, fast in calculation speed, and accurate in results. Compared with the existing calibration cloth calibration method, it is less affected by the environment and can achieve camera extrinsic parameter calibration more accurately and quickly; in addition, it can also allow implementation personnel to complete camera calibration quickly and efficiently, avoiding the situation where the workload of implementation personnel increases due to frequent installation of equipment.

Description

Technical Field

[0001] The present invention relates to the field of road engineering, and in particular to a camera extrinsic parameter calibration method and system based on a line laser device. Background Art

[0002] At present, the camera must be calibrated for external parameters every time the vehicle-mounted automation equipment in the field of road detection is installed. The camera external parameter calibration is to obtain the top-down transformation matrix to project the pixel coordinates to the world coordinates. The main calibration method currently uses the checkerboard calibration method, that is, placing a checkerboard calibration cloth on the road surface, manually or automatically identifying the pixel coordinates of the checkerboard corners, and presetting the world coordinates according to the calibration cloth checkerboard size information. The pixel coordinates of the checkerboard corners and the world coordinates are obtained by calculating the camera external parameters through the PnP algorithm. For accurate calibration, the camera model is constrained when the camera sees that the calibration target fills most of the image. In layman's terms, if a small calibration board is used, many combinations of camera parameters can explain the observed image, so in actual use, a large calibration cloth will be used as much as possible for calibration. Because the calibration cloth is too large, the cost is high, and it is affected by factors such as weather and personnel operation, the convenience of on-site implementation is greatly reduced, and it may affect the accuracy of calibration.

[0003] There are two main problems with the existing solution. First, subjectivity and inconsistency: the result of manual placement of the calibration cloth may be affected by the subjective judgment and quality of the personnel, resulting in inconsistent and inaccurate external parameter results. Second, poor feasibility: the use of a large calibration cloth to ensure accuracy will make it inconvenient to carry, and it will be affected by many external factors when placed manually on site, making it inconvenient to use. Summary of the invention

[0004] The purpose of the present invention is to provide a camera extrinsic calibration method and system based on a line laser device to solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A camera extrinsic calibration method based on a line laser device, the method comprising:

[0007] Setting parameters for the vehicle-mounted equipment; the vehicle-mounted equipment includes a vehicle-mounted camera and a line laser device; the parameters to be set in the vehicle-mounted camera include clarity, and the parameters to be set in the line laser device include power and emission position;

[0008] Sending a transmission instruction to the line laser device according to a preset first frequency, and sending a collection instruction to the vehicle-mounted camera according to a preset second frequency; wherein the first frequency is higher than the second frequency;

[0009] Receiving an image captured by a vehicle-mounted camera, identifying the image, and determining pixel coordinates;

[0010] The camera is calibrated externally according to the pixel coordinates.

[0011] As a further solution of the present invention: the vehicle-mounted camera is connected to a power supply device, and the power supply device includes a central industrial control computer and a vehicle-mounted power supply; the camera pixel of the vehicle-mounted camera is not less than 2 million pixels.

[0012] As a further solution of the present invention: the line laser equipment includes four straight line laser pens of the same specifications and a triangular bracket group, the power of the line laser emitter is ≥1W, and the length of the emitted line laser is L ≥4m; after the triangular bracket is unfolded, the height h is fixed, and there is an umbrella-shaped structure on the top, and there are four grooves in mutually perpendicular directions above the umbrella surface for installing the laser emitter, and the angle between the umbrella surface and the vertical line is arctan (2*h / L).

[0013] As a further solution of the present invention: the step of receiving the image captured by the vehicle-mounted camera, identifying the image, and determining the pixel coordinates includes:

[0014] Receive the image captured by the vehicle-mounted camera, and obtain the pixel coordinate sets of four laser lines based on the preset instance segmentation neural network algorithm; the number of elements in the pixel coordinate set of each laser line is not less than 64;

[0015] Perform a linear regression fit on each pixel coordinate set and calculate the coefficient of determination;

[0016] The determination coefficient is compared with a preset coefficient threshold, and when the determination coefficient is not less than 0.95, a calibration instruction is generated; when the determination coefficient is less than 0.95, a recalibration instruction is generated.

[0017] As a further solution of the present invention: the step of calibrating the camera external parameters according to the pixel coordinates includes:

[0018] Read the linear equations corresponding to the four laser lines in the image recognition result and calculate the coordinates of the intersection point;

[0019] Query the world coordinates of the intersection point; wherein the world coordinates are the four vertices of the rectangle;

[0020] The camera is calibrated externally according to the intersection coordinates and world coordinates.

[0021] The technical solution of the present invention also provides a camera extrinsic calibration system based on a line laser device, the system comprising:

[0022] A parameter setting module, used to set parameters for the vehicle-mounted equipment; the vehicle-mounted equipment includes a vehicle-mounted camera and a line laser device; the parameters to be set in the vehicle-mounted camera include clarity, and the parameters to be set in the line laser device include power and emission position;

[0023] An instruction sending module, used to send a transmission instruction to the line laser device according to a preset first frequency, and send a collection instruction to the vehicle-mounted camera according to a preset second frequency; wherein the first frequency is higher than the second frequency;

[0024] An image recognition module is used to receive an image captured by a vehicle-mounted camera, recognize the image, and determine pixel coordinates;

[0025] The calibration execution module is used to calibrate the external parameters of the camera according to the pixel coordinates.

[0026] As a further solution of the present invention: the vehicle-mounted camera is connected to a power supply device, and the power supply device includes a central industrial control computer and a vehicle-mounted power supply; the camera pixel of the vehicle-mounted camera is not less than 2 million pixels.

[0027] As a further solution of the present invention: the line laser equipment includes four straight line laser pens of the same specifications and a triangular bracket group, the power of the line laser emitter is ≥1W, and the length of the emitted line laser is L ≥4m; after the triangular bracket is unfolded, the height h is fixed, and there is an umbrella-shaped structure on the top, and there are four grooves in mutually perpendicular directions above the umbrella surface for installing the laser emitter, and the angle between the umbrella surface and the vertical line is arctan (2*h / L).

[0028] As a further solution of the present invention: the image recognition module includes:

[0029] A coordinate set creation unit, used for receiving an image captured by the vehicle-mounted camera, and obtaining pixel coordinate sets of four laser lines based on a preset instance segmentation neural network algorithm; the number of elements in the pixel coordinate set of each laser line is not less than 64;

[0030] A linear fitting unit is used to perform a linear regression fit on each pixel coordinate set and calculate the coefficient of determination;

[0031] The data comparison unit is used to compare the determination coefficient with a preset coefficient threshold value, and when the determination coefficient is not less than 0.95, a calibration instruction is generated; when the determination coefficient is less than 0.95, a recalibration instruction is generated.

[0032] As a further solution of the present invention: the calibration execution module includes:

[0033] An intersection coordinate calculation unit, used to read the linear equations corresponding to the four laser lines in the image recognition result and calculate the intersection coordinates;

[0034] A world coordinate query unit, used to query the world coordinates of the intersection; wherein the world coordinates are the four vertices of the rectangle;

[0035] The processing execution unit is used to calibrate the external parameters of the camera according to the intersection coordinates and the world coordinates.

[0036] Compared with the prior art, the beneficial effects of the present invention are: the present invention is simple to operate, fast in calculation speed, and accurate in results. Compared with the existing calibration cloth calibration method, the present invention is less affected by the environment and can achieve camera extrinsic calibration more accurately and quickly. In addition, it can also allow implementers to complete camera calibration quickly and efficiently, avoiding the situation where the workload of implementers is increased due to frequent installation of equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention.

[0038] Figure 1 The flowchart of the camera extrinsic calibration method based on line laser equipment. DETAILED DESCRIPTION

[0039] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0040] Figure 1 A flowchart of a camera extrinsic calibration method based on a line laser device is shown. In an embodiment of the present invention, a camera extrinsic calibration method based on a line laser device includes:

[0041] Setting parameters for the vehicle-mounted equipment; the vehicle-mounted equipment includes a vehicle-mounted camera and a line laser device; the parameters to be set in the vehicle-mounted camera include clarity, and the parameters to be set in the line laser device include power and emission position;

[0042] Sending a transmission instruction to the line laser device according to a preset first frequency, and sending a collection instruction to the vehicle-mounted camera according to a preset second frequency; wherein the first frequency is higher than the second frequency;

[0043] Receiving an image captured by a vehicle-mounted camera, identifying the image, and determining pixel coordinates;

[0044] The camera is calibrated externally according to the pixel coordinates.

[0045] In one example of the technical solution of the present invention, a vehicle-mounted camera, a central industrial control computer and other equipment are installed on a vehicle, wherein the central industrial control computer is powered by the vehicle, the vehicle-mounted camera is connected to the central industrial control computer, and the central industrial control computer is manually operated to send collection instructions and complete calculations.

[0046] Specifically, the implementation process is summarized as follows:

[0047] (1) After the system is started, place the line laser device within the camera field of view, and try to keep the bottom edge of the laser line parallel to the bottom edge of the image.

[0048] (2) After the line laser equipment is placed, the implementation personnel issue acquisition instructions on the central industrial control computer. After the central industrial control computer receives the acquired images, it will call the model to calculate the external parameters.

[0049] (3) The system records the camera external parameters locally and uploads them to the cloud system.

[0050] As a preferred embodiment of the technical solution of the present invention, the vehicle-mounted camera is connected to a power supply device, and the power supply device includes a central industrial control computer and a vehicle-mounted power supply; the camera pixel of the vehicle-mounted camera is not less than 2 million pixels.

[0051] In addition, the line laser equipment includes four straight line laser pens of the same specifications and a triangular bracket group. The power of the line laser emitter is ≥1W, and the length of the emitted line laser is L≥4m. After the triangular bracket is unfolded, the height h is fixed, and there is an umbrella-shaped structure on the top. There are four grooves in perpendicular directions above the umbrella surface for installing the laser emitter. The angle between the umbrella surface and the vertical line is arctan(2*h / L).

[0052] In one example of the technical solution of the present invention, the vehicle-mounted intelligent terminal includes: a vehicle-mounted camera, a central industrial control computer and a line laser device, wherein:

[0053] (1) The vehicle-mounted camera (with known camera internal parameters) is powered by a central industrial computer or by the vehicle. The camera has a pixel of ≥ 2 million pixels and has clear image quality when the vehicle is in motion. It is installed outside the vehicle and can be used to collect road surface images. The collected images can clearly distinguish the road surface texture.

[0054] (2) The central industrial computer is installed in the vehicle and powered by the vehicle. The central industrial computer has a 4G / 5G mobile Internet module, a high-precision positioning module, and corresponding acquisition software to send instructions to the vehicle camera to collect images. The central industrial computer can include a GPU processing unit, and the neural network program can use the GPU processing unit to achieve prediction acceleration.

[0055] (3) The line laser equipment consists of four straight line laser pens of the same specifications and a triangular bracket. The power of the line laser emitter is ≥1W, and the length of the emitted line laser is L≥4m. After the triangular bracket is unfolded, the height h is fixed. There is an umbrella-shaped structure on the top. There are four mutually perpendicular grooves on the top of the umbrella surface for installing laser emitters. The angle between the umbrella surface and the vertical line is arctan (2*h / L). Since the height of the triangular bracket is fixed, the distance between each laser emitter and the laser line and the angle of the road surface are consistent, so that the four laser lines form a standard square on the road surface.

[0056] As a preferred embodiment of the technical solution of the present invention, the step of receiving an image captured by a vehicle-mounted camera, identifying the image, and determining pixel coordinates includes:

[0057] Receive the image captured by the vehicle-mounted camera, and obtain the pixel coordinate sets of four laser lines based on the preset instance segmentation neural network algorithm; the number of elements in the pixel coordinate set of each laser line is not less than 64;

[0058] Perform a linear regression fit on each pixel coordinate set and calculate the coefficient of determination;

[0059] The determination coefficient is compared with a preset coefficient threshold, and when the determination coefficient is not less than 0.95, a calibration instruction is generated; when the determination coefficient is less than 0.95, a recalibration instruction is generated.

[0060] Furthermore, the step of calibrating the camera external parameters according to the pixel coordinates includes:

[0061] Read the linear equations corresponding to the four laser lines in the image recognition result and calculate the coordinates of the intersection point;

[0062] Query the world coordinates of the intersection point; wherein the world coordinates are the four vertices of the rectangle;

[0063] The camera is calibrated externally according to the intersection coordinates and world coordinates.

[0064] In an example of the technical solution of the present invention, a specific camera calibration analysis method is provided as follows:

[0065] After fixing the camera, place the line laser equipment in the camera field of view, use the central industrial control computer to collect pictures, and use the example segmentation neural network algorithm to obtain the pixel coordinates of the four laser lines. The number of pixel coordinates of each line n_points≥64. Perform linear regression fitting on each laser line, and then calculate the coefficient of determination (R2). If R2≥0.95, it is considered that during calibration, this line is irradiated on a horizontal plane, and the camera external parameter calculation can be performed. Otherwise, it is considered that there is a problem in the on-site implementation, resulting in the bending or obstruction of the laser line, and it is necessary to remind the camera to recalibrate. If there is no problem with the four lasers, the pixel coordinates of the intersections (A, B, C, D) between the laser lines can be solved according to the linear regression equation. Since the size of the square formed by the line laser is L*L, assuming that the world coordinates of point A are (0, 0, 0), then the world coordinates of the four intersections can be obtained as follows: (0, 0, 0), (0, L, 0), (L, 0, 0), (L, L, 0). At this point we use Python's opencv library to solve the PnP algorithm and get the camera's external parameters at this time.

[0066] As a preferred embodiment of the technical solution of the present invention, the technical solution of the present invention also provides a camera extrinsic calibration system based on a line laser device, the system comprising:

[0067] A parameter setting module, used to set parameters for the vehicle-mounted equipment; the vehicle-mounted equipment includes a vehicle-mounted camera and a line laser device; the parameters to be set in the vehicle-mounted camera include clarity, and the parameters to be set in the line laser device include power and emission position;

[0068] An instruction sending module, used to send a transmission instruction to the line laser device according to a preset first frequency, and send a collection instruction to the vehicle-mounted camera according to a preset second frequency; wherein the first frequency is higher than the second frequency;

[0069] An image recognition module is used to receive an image captured by a vehicle-mounted camera, recognize the image, and determine pixel coordinates;

[0070] The calibration execution module is used to calibrate the external parameters of the camera according to the pixel coordinates.

[0071] Wherein, the vehicle-mounted camera is connected to a power supply device, and the power supply device includes a central industrial control computer and a vehicle-mounted power supply; the camera pixel of the vehicle-mounted camera is not less than 2 million pixels.

[0072] The line laser device includes four straight line laser pens of the same specifications and a triangular bracket group. The power of the line laser emitter is ≥1W, and the length of the emitted line laser is L ≥4m. After the triangular bracket is unfolded, the height h is fixed, and there is an umbrella-shaped structure on the top. There are four grooves in perpendicular directions above the umbrella surface for installing the laser emitter. The angle between the umbrella surface and the vertical line is arctan (2*h / L).

[0073] Furthermore, the image recognition module includes:

[0074] A coordinate set creation unit, used for receiving an image captured by the vehicle-mounted camera, and obtaining pixel coordinate sets of four laser lines based on a preset instance segmentation neural network algorithm; the number of elements in the pixel coordinate set of each laser line is not less than 64;

[0075] A linear fitting unit is used to perform a linear regression fit on each pixel coordinate set and calculate the coefficient of determination;

[0076] The data comparison unit is used to compare the determination coefficient with a preset coefficient threshold value, and when the determination coefficient is not less than 0.95, a calibration instruction is generated; when the determination coefficient is less than 0.95, a recalibration instruction is generated.

[0077] Specifically, the calibration execution module includes:

[0078] An intersection coordinate calculation unit, used to read the linear equations corresponding to the four laser lines in the image recognition result and calculate the intersection coordinates;

[0079] A world coordinate query unit, used to query the world coordinates of the intersection; wherein the world coordinates are the four vertices of the rectangle;

[0080] The processing execution unit is used to calibrate the external parameters of the camera according to the intersection coordinates and the world coordinates.

[0081] The method mentioned in this application is a camera extrinsic parameter calibration method based on a line laser device and a convolutional neural network. It utilizes a line laser device and a target detection neural network algorithm to quickly and accurately complete the camera extrinsic parameter calibration.

[0082] In terms of execution efficiency, this method is simple to operate, fast in calculation, and accurate in results. Compared with the existing calibration cloth calibration method, it is less affected by the environment and can achieve camera extrinsic calibration more accurately and quickly.

[0083] In terms of economic benefits, line laser equipment has been increasingly used in engineering in recent years, and there are many products on the market. Compared with larger-sized calibration cloths, they have lower costs. The method mentioned in this patent not only reduces the existing costs, but also greatly improves the convenience and accuracy of camera external parameter calibration.

[0084] In terms of social benefits, the method mentioned in this patent allows implementers to complete camera calibration quickly and efficiently, avoiding the situation where the workload of implementers increases due to frequent equipment installation.

[0085] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A camera extrinsic calibration method based on a line laser device, characterized in that: The method comprises: Setting parameters for the vehicle-mounted equipment; the vehicle-mounted equipment includes a vehicle-mounted camera and a line laser device; the parameters to be set in the vehicle-mounted camera include clarity, and the parameters to be set in the line laser device include power and emission position; Sending a transmission instruction to the line laser device according to a preset first frequency, and sending a collection instruction to the vehicle-mounted camera according to a preset second frequency; wherein the first frequency is higher than the second frequency; Receiving an image captured by a vehicle-mounted camera, identifying the image, and determining pixel coordinates; Performing extrinsic calibration on the camera according to the pixel coordinates; The line laser device comprises four straight line laser pens of the same specifications and a triangular bracket set. The power of the line laser emitter is ≥1W, and the length of the emitted line laser is L ≥4m. After the triangular bracket is unfolded, the height h is fixed, and there is an umbrella-shaped structure on the top. There are four mutually perpendicular grooves on the top of the umbrella surface for installing the laser emitter. The angle between the umbrella surface and the vertical line is arctan (2*h / L); The step of receiving an image captured by a vehicle-mounted camera, identifying the image, and determining pixel coordinates comprises: Receive the image captured by the vehicle-mounted camera, and obtain the pixel coordinate sets of four laser lines based on the preset instance segmentation neural network algorithm; the number of elements in the pixel coordinate set of each laser line is not less than 64; Perform a linear regression fit on each pixel coordinate set and calculate the coefficient of determination; The determination coefficient is compared with a preset coefficient threshold, and when the determination coefficient is not less than 0.95, a calibration instruction is generated; when the determination coefficient is less than 0.95, a recalibration instruction is generated; The step of calibrating the camera's external parameters according to the pixel coordinates comprises: Read the linear equations corresponding to the four laser lines in the image recognition result and calculate the coordinates of the intersection point; Query the world coordinates of the intersection point; wherein the world coordinates are the four vertices of the rectangle; The camera is calibrated externally according to the intersection coordinates and world coordinates.

2. The camera extrinsic calibration method based on a line laser device according to claim 1, characterized in that: The vehicle-mounted camera is connected to a power supply device, which includes a central industrial control computer and a vehicle-mounted power supply; the camera pixel of the vehicle-mounted camera is not less than 2 million pixels.

3. A camera extrinsic calibration system based on a line laser device, characterized in that: The system comprises: A parameter setting module, used to set parameters for the vehicle-mounted equipment; the vehicle-mounted equipment includes a vehicle-mounted camera and a line laser device; the parameters to be set in the vehicle-mounted camera include clarity, and the parameters to be set in the line laser device include power and emission position; An instruction sending module, used to send a transmission instruction to the line laser device according to a preset first frequency, and send a collection instruction to the vehicle-mounted camera according to a preset second frequency; wherein the first frequency is higher than the second frequency; An image recognition module is used to receive an image captured by a vehicle-mounted camera, recognize the image, and determine pixel coordinates; A calibration execution module, used for calibrating the camera's external parameters according to the pixel coordinates; The line laser device comprises four straight line laser pens of the same specifications and a triangular bracket set. The power of the line laser emitter is ≥1W, and the length of the emitted line laser is L ≥4m. After the triangular bracket is unfolded, the height h is fixed, and there is an umbrella-shaped structure on the top. There are four mutually perpendicular grooves on the top of the umbrella surface for installing the laser emitter. The angle between the umbrella surface and the vertical line is arctan (2*h / L); The image recognition module comprises: A coordinate set creation unit, used for receiving an image captured by the vehicle-mounted camera, and obtaining pixel coordinate sets of four laser lines based on a preset instance segmentation neural network algorithm; the number of elements in the pixel coordinate set of each laser line is not less than 64; A linear fitting unit is used to perform a linear regression fit on each pixel coordinate set and calculate the coefficient of determination; A data comparison unit, used to compare the determination coefficient with a preset coefficient threshold, and generate a calibration instruction when the determination coefficient is not less than 0.95; and generate a recalibration instruction when the determination coefficient is less than 0.95; The calibration execution module comprises: An intersection coordinate calculation unit, used to read the linear equations corresponding to the four laser lines in the image recognition result and calculate the intersection coordinates; A world coordinate query unit, used to query the world coordinates of the intersection; wherein the world coordinates are the four vertices of the rectangle; The processing execution unit is used to calibrate the external parameters of the camera according to the intersection coordinates and the world coordinates.

4. The camera extrinsic calibration system based on a line laser device according to claim 3, characterized in that: The vehicle-mounted camera is connected to a power supply device, which includes a central industrial control computer and a vehicle-mounted power supply; the camera pixel of the vehicle-mounted camera is not less than 2 million pixels.

Images