A calibration method for camera calibration on unmanned trucks

Abstract

This invention relates to the field of lane line edge detection, and provides a calibration correction method for unmanned truck cameras. The method acquires trajectory information, confirms the coordinate system through a positioning system, and transforms the vehicle coordinates to mark the driving section. Then, based on LiDAR, it identifies and tracks guardrails to obtain the corresponding guardrail lines, calculates the actual position and attitude information of the current vehicle, compares the visual lane line position, and utilizes a method that allows online calibration of camera calibration parameters. This enables online calibration of the camera, avoiding camera malfunctions due to abnormal conditions and improving detection accuracy.

Description

Technical Field

[0001] This invention relates to the field of lane line edge detection, and more specifically to a calibration method for unmanned truck cameras. Background Technology

[0002] Today, port transportation is a crucial guarantee for port operations, and efficient handling can increase port throughput. With development, unmanned container trucks are now used in ports, where cameras play a vital role in their operation. These cameras can identify vehicles, pedestrians, cones, and other obstacles near the unmanned truck, and also provide lane marking assistance. However, due to the inherent characteristics of container trucks, varying cargo loads during transport can cause the tractor unit to sink and deform to different degrees, leading to changes in the external parameters of the cameras mounted on the tractor unit and resulting in inaccurate detection. Furthermore, in various complex environments, the calibration accuracy of the cameras can be affected by various factors, such as bracket distortion, vehicle load, and vehicle tilt, which will impact the accurate visual identification of obstacle positions.

[0003] Therefore, this invention proposes a calibration method for unmanned truck cameras, which uses lidar to correct the recognition of lane lines by the camera, thus achieving real-time calibration. Summary of the Invention

[0004] The purpose of this invention is to provide a calibration method for unmanned truck cameras, which aims to improve the problem that the tractor unit of an unmanned truck may sink or deform to varying degrees due to different loads, which in turn causes changes in the external parameters of the camera installed on the tractor unit, resulting in inaccurate detection.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: a calibration method for unmanned truck cameras, comprising the following steps:

[0006] Unmanned trucks acquire trajectory information of their driving routes;

[0007] The positioning system records and saves the marking information of the unmanned truck when it travels according to the trajectory information, and converts the marking information into a coordinate system;

[0008] The guardrails on the road section are identified by LiDAR, and a guardrail line is selected and fitted based on the point cloud data of the identified guardrails.

[0009] The visual lane lines are obtained by using a camera to capture the fitted lane lines on both sides of the road segment and verifying the fitted lane lines on both sides based on the fitted guardrail lines.

[0010] The height values ​​of the camera's rotation and translation vectors are iteratively corrected online, and a threshold constraint is applied to obtain the corrected rotation and translation vector values.

[0011] The visual lane lines are verified based on the corrected rotation vector values ​​and translation vector values ​​to obtain the corrected visual lane lines.

[0012] Preferably, the coordinate system is a UTM coordinate system or a latitude and longitude coordinate system.

[0013] Preferably, the vehicle body coordinate system is obtained according to the coordinate system transformation, and the first straight line of the unmanned truck is fitted according to the least squares method;

[0014] A threshold is manually set to filter point cloud data, and the distance between the filtered point cloud data and the first straight line is calculated. All point cloud data that are less than the mechanical threshold are collected to obtain a point set.

[0015] The point set is proportionally divided into three segments: the first segment, the middle segment, and the last segment. A point cloud is selected from the point set of each segment to obtain the second straight line between the first segment and the middle segment or between the last segment and the middle segment, as well as the third straight line between the first segment and the last segment.

[0016] The fourth line is obtained by calculating a weighted average of the second and third lines.

[0017] Traverse the remaining point cloud within the point set and calculate the average distance from the remaining point cloud to the fourth line;

[0018] Discard point clouds whose distance between the two endpoints of the fourth straight line is less than 30 meters, and calculate the ratio R of point clouds within the qualified range to point clouds in the point set.

[0019] If the ratio R is greater than or equal to 0.8, perform at least 100 repeated iterations to obtain the fifth straight line of the final guardrail line; if the ratio R is less than 0.8, skip the current iteration.

[0020] Based on the slope and intercept of the fifth line, Kalman tracking stability testing is performed to obtain the final expression for the laser guardrail line:

[0021] y = k fence ×x+b fence k is the slope and b is the intercept.

[0022] Preferably, the angle between the heading of the unmanned truck and the first straight line is obtained, and the guardrail point cloud is manually filtered when the angle is less than or equal to 15°.

[0023] Preferably, if the ratio R is greater than 0.98, then the fourth straight line is directly determined as the fifth straight line of the final guardrail line.

[0024] Preferably, the trajectory information is a straight road. The image information of the driving section is obtained by a camera, and the left lane line point set P1 and the right lane line point set P2 of the vehicle coordinate system are fitted by the UFLD model.

[0025] Select any two point clouds in P1 and obtain the sixth straight line connecting the two point clouds.

[0026] Calculate the distance from the remaining point cloud in P1 to the sixth line, and select the point cloud within 0.2 meters to obtain the candidate point cloud set, and obtain the average distance of the point cloud within the candidate point cloud set;

[0027] Obtain the proportion M of the candidate point cloud set to all point clouds in P1. If the proportion M is greater than or equal to 0.9, perform multiple iterative calculations; if the proportion M is less than 0.9, recalculate by selecting any two point clouds in P1.

[0028] After multiple iterations of calculation, the sixth straight line with the smallest average value is selected as the left lane line, and the slope and intercept values ​​are obtained by least squares method based on the cluster of candidate points.

[0029] Similarly, calculate the seventh line of the P2 point set, as well as the slope and intercept of the seventh line, to obtain the right lane line;

[0030] Based on the expression for the final laser guardrail line, the calculation formulas for the sixth and seventh straight lines are obtained:

[0031]

[0032] Preferably, if the ratio value M is greater than or equal to 1, then the sixth straight line in this calculation is directly determined as the left lane line.

[0033] Preferably, when performing multiple iterations, the number of iterations does not exceed 50, and the angle difference between lane lines on the same side of consecutive frames is calculated using the following formula:

[0034]

[0035] Determine if the angle value is less than or equal to 2° for five consecutive times. If so, perform track tracking of the unmanned container truck; otherwise, recalculate.

[0036] Preferably, x1 and x2 are selected as adjustment points on the visual lane line, and x2-x1≥10, x1≥0, and the vehicle width calculation formula W for the unmanned truck is... camera :

[0037]

[0038]

[0039] When acquiring images through a camera, the values ​​of the rotation vectors pitch, yaw, and roll, as well as the translation vector variable value Z, are obtained.

[0040] Set the pitch deviation angle range to within 0.5°, and determine whether the visual lane line slope satisfies the following formula:

[0041] Where PI = π;

[0042] If this condition is not met, then the initial correction angle value AngStep is selected as 5°, and if... and pitch cur =pitch pre -AngStep, or, and pitch cur =pitch pre When +AngStep is used, the visual lane line formula formed by the transformation of the camera's intrinsic and extrinsic parameters is re-acquired and calculated; and the value of AngStep is recalibrated to half of the previous value each time.

[0043] If satisfied, no correction is needed or pitch correction is complete;

[0044] Set the yaw deviation angle range to within 0.5°, and determine whether the slope of the visual lane line satisfies the following formula:

[0045]

[0046] If this condition is not met, then the initial correction angle value AngStep is selected as 5°, and if... And yaw cur =yaw pre -AngStep, or, And yaw cur =yaw pre +AngStep then recalculates the visual lane line formula formed by the transformation of the camera's intrinsic and extrinsic parameters; and each time the AngStep value is recalibrated, it is halved from the previous value.

[0047] If satisfied, no correction is needed or the yaw correction is completed.

[0048] Set a roll deviation threshold of 0.05m or less, and determine whether the visual lane line slope satisfies the following formula:

[0049]

[0050] If this condition is not met, then the initial correction angle value AngStep is selected as 5°, and if... And roll cur =roll pre -AngStep, or, And roll cur =roll pre +AngStep then recalculates the visual lane line formula formed by the transformation of the camera's intrinsic and extrinsic parameters; and each time the AngStep value is recalibrated, it is halved from the previous value.

[0051] If satisfied, no correction is required or the roll correction is completed.

[0052] Lane width W is obtained through location information. true And set the correction threshold to 0.1 meters, and determine whether the following formula is satisfied:

[0053]

[0054] If this condition is not met, then an initial correction height value of HighStep = 0.5 is selected; and, and Z cur =Z pre -HightStep, or, and Z cur =Z pre During +HightStep, the visual lane line formula formed by the conversion of the camera's intrinsic and extrinsic parameters is re-acquired and calculated; and the value of HighStep is recalibrated to half of the previous value each time.

[0055] If the conditions are met, no correction is required or Z-correction is completed.

[0056] Preferably, the values ​​of pitch, yaw, and roll of the rotation vector and the value of Z of the translation vector are corrected at least S times, and S≥100, and the average value of pitch, yaw, roll, or Z obtained by S corrections is calculated as the final correction value.

[0057] Each time the camera is calibrated and the final calibration value is obtained for the first time, it is counted as one round of calculation. After repeating the calculation for M rounds, the proportion T of the total number of camera calibrations that need to be performed is calculated.

[0058] If T is greater than 0.2, then the statistical correction should be restarted.

[0059] If T is less than or equal to 0.2, then the counting of round M is restarted, and the pitch value is assigned as a multiple of N, where 3 > N > 1.

[0060] By adopting the above technical solution, the present invention has the following advantages compared with the prior art:

[0061] 1. After acquiring trajectory information, the system confirms the coordinate system and transforms the vehicle coordinates to mark the driving segment. Then, based on LiDAR, it identifies and tracks the guardrail to obtain the corresponding guardrail lines. It calculates the actual position and attitude information of the vehicle body, compares it with the visual lane line position, and uses a method that allows online calibration of camera parameters to perform online calibration. This avoids camera malfunctions caused by abnormal conditions and improves detection accuracy. Attached Figure Description

[0062] Figure 1 This is a flowchart of the calibration method for unmanned truck cameras described in this invention. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0064] Additionally, it should be noted that the terms "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer" are all based on the orientation or positional relationship shown in the accompanying drawings. They are merely for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element of the present invention must have a specific orientation. Therefore, they should not be construed as limitations on the present invention.

[0065] When an element is referred to as being "fixed to," "set on," or "contained on" another element, it can be directly on or indirectly on that other element. When an element is referred to as being "connected to," it can be directly connected to or indirectly connected to that other element.

[0066] Unless otherwise expressly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication between two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0067] Example

[0068] Please refer to Figure 1 As shown, this embodiment provides a calibration method for unmanned truck cameras, including the following steps:

[0069] S101, Unmanned trucks acquire trajectory information of their driving routes.

[0070] Long, straight sections of road in ports are where unmanned trucks frequently travel. Therefore, ensuring that unmanned trucks can travel along these long, straight routes and improving safety is crucial. Thus, the travel trajectory information, representing these long, straight sections, is transmitted via a wireless network (such as a 5G network) and stored in the unmanned truck.

[0071] Specifically, taking a straight route with corresponding guardrails along its sides and planned lane lines parallel to the guardrails as an example, and assuming the road section is at least 100 meters long, when an unmanned truck enters this area, the GNSS (Global Navigation Satellite System) positioning system can accurately locate the truck's position and simultaneously activate the online calibration function.

[0072] S102. Use a positioning system to record and save the marking information of the unmanned truck when it travels according to the trajectory information, and convert the marking information into a coordinate system.

[0073] During operation, the unmanned truck is positioned using a positioning system. The wheels of the truck closest to the guardrail are positioned precisely on the lane line, allowing the truck to travel in a straight line along that lane until it exits the section without guardrails. During this process, the positioning data (GNSS) is recorded, and the section is marked and saved. The saved format can be a UTM coordinate system (Universal Transverse Carrier Grid System, a Cartesian coordinate system widely used in topographic maps, as a reference grid for satellite imagery and natural resource databases, and for other applications requiring precise positioning), or a latitude and longitude coordinate system. Alternatively, positioning can be achieved by marking positioning points; for example, in a 400-meter-long storage yard, a key positioning point can be recorded every 20 meters along the guardrail. Simultaneously, this invention pre-sets the lane width to 2.8 meters and the guardrail to be 0.5 meters from the lane edge.

[0074] Of course, a simple, specific positioning system can also be used to record several key positioning points along the route. For example, in a 400m port area, a key positioning point can be recorded directly at the guardrail location every 20 meters. Simultaneously, the width of the lane line can be artificially limited to 2.8 meters, with the guardrail 0.5 meters from the edge of the lane line, and tests can be conducted based on the data from this lane line.

[0075] S103. Identify the guardrails on the road section using lidar, and select and fit a guardrail line based on the point cloud data of the identified guardrails.

[0076] Specifically, the vehicle's coordinate system is obtained through the coordinate system transformation, and the first straight line of the unmanned truck is fitted using the least squares method. The least squares method is a mathematical optimization technique. It finds the best function match for the data by minimizing the sum of squared errors. The least squares method can be used to easily obtain unknown data while minimizing the sum of squared errors between the obtained data and the actual data. The least squares method can also be used for curve fitting. Other optimization problems can also be expressed using the least squares method by minimizing energy or maximizing entropy. The calculation formula for the least squares method is existing and will not be shown here; the same applies to the least squares method described below.

[0077] Point cloud data is filtered by manually setting thresholds, and the distance between the filtered point cloud data and the first straight line is calculated. All point cloud data with distances less than the mechanical threshold are aggregated to obtain a point set. The thresholds can be set as follows: ±5 meters for the y-axis, 0-2 meters for the z-axis, and 0-50 meters for the x-axis. Specifically, point cloud data within these ranges are selected. The mechanical threshold can be adjusted appropriately based on different laser voxelization accuracies; a preferred threshold is 0.3 meters.

[0078] The point set is proportionally divided into three segments: the first segment, the middle segment, and the last segment. A point cloud is selected from the point set of each segment to obtain the second straight line between the first segment and the middle segment or the last segment and the middle segment, as well as the third straight line between the first segment and the last segment. Specifically, the point set is selected within a 50-meter range on the x-axis, so the length of the point set is 50 meters. Therefore, the range [0, 0.25) can be divided into the first segment, [0.25, 0.75) into the middle segment, and [0.75, 1] ​​into the last segment. Taking 50 meters as an example, it is divided into the first segment 0-12.5 meters, the middle segment 12.5-37.5 meters, and the last segment 37.5-50 meters.

[0079] The fourth line is obtained by calculating the weighted average of the second and third lines.

[0080] The average weighted calculation for the second and third lines is based on the slope k2 and intercept b2 of the second line, and the slope k3 and intercept b3 of the third line. The calculation formula is as follows:

[0081]

[0082] The slope k4 and intercept b4 of the fourth line are calculated in this way, thus obtaining the fourth line.

[0083] Traverse the remaining point cloud within the point set and calculate the average distance from the remaining point cloud to the fourth line; discard point clouds whose distance between the two endpoints of the fourth line is less than 30 meters, and calculate the ratio R of the point cloud within the qualified range to the point cloud in the point set.

[0084] If the calculated point cloud distances at both endpoints of the fourth straight line are less than 30 meters, it indicates that the original point cloud data did not capture complete guardrail data; in other words, the complete guardrail is not visible. Therefore, the data for this frame should be discarded. The solution is simply to recalculate the data for the next frame until the required data is obtained.

[0085] At this point, the scale value R is calculated. Once the guardrail line is obtained, the point cloud belonging to the guardrail line is considered a qualified point cloud, that is, the point cloud within the mechanical threshold range of 0.3 meters mentioned above. Thus, the qualified point cloud and the point cloud in the point set are statistically compared to calculate the scale value R.

[0086] If the ratio R is greater than or equal to 0.8, perform at least 100 repeated iterations to obtain the fifth straight line of the final guardrail line; if the ratio R is less than 0.8, skip the current iteration.

[0087] Based on the slope and intercept of the fifth line, Kalman tracking stability testing is performed to obtain the final expression for the laser guardrail line:

[0088] y = k fence ×x+b fence k is the slope and b is the intercept.

[0089] Furthermore, in this embodiment, after fitting the first straight line, the angle between the heading of the unmanned truck and the first straight line can be obtained, and the guardrail point cloud can be manually filtered when the angle is less than or equal to 15°.

[0090] When the angle between the unmanned truck's cab and the first straight line is less than or equal to 15°, the truck is considered to be in a correct posture, and subsequent operations can proceed to ensure the safety and efficiency of these operations. Conversely, if the angle between the truck's cab and the first straight line is greater than 15°, the truck's posture is adjusted until a frame indicating an angle of less than or equal to 15° is obtained, at which point the posture adjustment stops.

[0091] Furthermore, in this embodiment, if the ratio value R is greater than 0.98, the fourth straight line is directly determined as the fifth straight line of the final guardrail line. That is, if the ratio value R is greater than 0.98, the calculation is more accurate and can be directly adopted. Therefore, the iterative calculation can be skipped directly to determine the fifth straight line of the final guardrail line.

[0092] S104. Obtain the fitted lane lines on both sides of the driving road segment through the camera, and verify the fitted lane lines on both sides based on the fitted guardrail lines to obtain the visual lane lines.

[0093] Specifically, since the trajectory information is a straight road, by acquiring image information of the driving section through the camera, the left lane line point set P1 and the right lane line point set P2 of the vehicle coordinate system can be fitted by the UFLD model.

[0094] Select any two point clouds in P1 to obtain the sixth straight line connecting the two point clouds;

[0095] Calculate the distance from the remaining point cloud in P1 to the sixth line, and select the point cloud within 0.2 meters to obtain the candidate point cloud set, and obtain the average distance of the point cloud within the candidate point cloud set;

[0096] Obtain the proportion M of the candidate point cloud set to all point clouds in P1. If the proportion M is greater than or equal to 0.9, perform multiple iterative calculations; if the proportion M is less than 0.9, recalculate by selecting any two point clouds in P1.

[0097] After multiple iterations of calculation, the sixth straight line with the smallest average value is selected as the left lane line, and the slope and intercept values ​​are obtained by least squares method based on the cluster of candidate points.

[0098] Similarly, calculate the seventh line of the P2 point set, as well as the slope and intercept of the seventh line, to obtain the right lane line;

[0099] Based on the expression for the final laser guardrail line, the calculation formulas for the sixth and seventh straight lines are obtained:

[0100]

[0101] Furthermore, if the ratio value M is greater than or equal to 1 in this step, then the sixth straight line calculated in this time is directly determined to be the left lane line. That is, if the ratio value M is equal to 1, it means that the candidate point cloud is the same as all point clouds. All point clouds are within 0.2 meters. This detection is a high-precision detection, so it can be directly determined to be the left lane line.

[0102] Furthermore, in this step, during multiple iterations, the number of iterations does not exceed 50, and the angle difference between lane lines on the same side of consecutive frames is calculated using the following formula:

[0103]

[0104] The system checks if the angle value is less than or equal to 2° five times consecutively. If so, it performs track tracking for the unmanned truck; otherwise, it recalculates. This process ensures accurate lane line data, improving the safety and efficiency of the unmanned truck's operation.

[0105] S105. Perform online iterative correction on the height values ​​of the camera's rotation vector and translation vector, and impose a threshold constraint to obtain the corrected rotation vector values ​​and translation vector variable values.

[0106] Specifically, x1 and x2 are selected as adjustment points on the visual lane line, and x2-x1≥10, x1≥0. The formula for calculating the width W of the unmanned truck is... camera :

[0107]

[0108] For example, we can select two points, x1 = 0 and x2 = 20, as adjustment points. By substituting them into the formula above, we can obtain four points of the left and right lane lines in the vehicle coordinate system, as well as the vehicle width value at this time. At this point, we can correct the variable values ​​of the rotation vector and translation vector.

[0109] When acquiring images through a camera, obtain the values ​​of the rotation vectors pitch, yaw, and roll, as well as the translation vector variable value Z;

[0110] Set the pitch deviation angle range to within 0.5°, and determine whether the visual lane line slope satisfies the following formula:

[0111] Where PI = π;

[0112] If this condition is not met, then the initial correction angle value AngStep is selected as 5°, and if... and pitch cur =pitch pre -AngStep, or, and pitch cur =pitch pre When +AngStep is used, the visual lane line formula formed by the transformation of the camera's intrinsic and extrinsic parameters is re-acquired and calculated; and the value of AngStep is recalibrated to half of the previous value each time.

[0113] If satisfied, no correction is needed or pitch correction is complete;

[0114] Set the yaw deviation angle range to within 0.5°, and determine whether the slope of the visual lane line satisfies the following formula:

[0115]

[0116] If this condition is not met, then the initial correction angle value AngStep is selected as 5°, and if... And yaw cur =yaw pre -AngStep, or, And yaw cur =yaw pre +AngStep then recalculates the visual lane line formula formed by the transformation of the camera's intrinsic and extrinsic parameters; and each time the AngStep value is recalibrated, it is halved from the previous value.

[0117] If satisfied, no correction is needed or yaw correction is completed.

[0118] Set a roll deviation threshold of 0.05m or less, and determine whether the visual lane line slope satisfies the following formula:

[0119]

[0120] If this condition is not met, then the initial correction angle value AngStep is selected as 5°, and if... And roll cur =roll pre -AngStep, or, and roll cur =roll pre +AngStep then recalculates the visual lane line formula formed by the transformation of the camera's intrinsic and extrinsic parameters; and each time the AngStep value is recalibrated, it is halved from the previous value.

[0121] If satisfied, no correction is required or the roll correction is completed.

[0122] Lane width W is obtained through location information. true And set the correction threshold to 0.1 meters, and determine whether the following formula is satisfied:

[0123]

[0124] If this condition is not met, then an initial correction height value of HighStep = 0.5 is selected; and, and Z cur =Z pre -HightStep, or, and Z cur =Z pre During +HightStep, the visual lane line formula formed by the conversion of the camera's intrinsic and extrinsic parameters is re-acquired and calculated; and the value of HighStep is recalibrated to half of the previous value each time.

[0125] If the conditions are met, no correction is required or Z-correction is completed.

[0126] In this step, the values ​​of pitch, yaw, and roll of the rotation vector and the value of Z of the translation vector are corrected at least S times, and S≥100, preferably 100 times, and the average value of pitch, yaw, roll, or Z obtained after S corrections is calculated as the final correction value.

[0127] Furthermore, an early termination signal can be set. Specifically, each time the camera is calibrated and the final calibration value is obtained for the first time, it is counted as one round of calculation. After repeating the calculation for M rounds, the proportion T of the total number of camera calibrations required is calculated. For ease of calculation, M can be set to 50 rounds.

[0128] If T is greater than 0.2, then the statistical correction should be restarted.

[0129] If T is less than or equal to 0.2, then the counting of M rounds is repeated, and the pitch value is assigned as a multiple of N, where 3 > N > 1, preferably 2. For example, if the original pitch threshold is 0.5°, then the constraint is 1°, the yaw constraint is 1°, the roll constraint is 0.1m, and the z constraint is 0.2m.

[0130] S106. Verify the visual lane lines based on the corrected rotation vector values ​​and translation vector variable values ​​to obtain the corrected visual lane lines.

[0131] Specifically, by providing a calibration method for unmanned truck cameras, based on laser radar for guardrail recognition and tracking, the actual position and attitude information of the current vehicle body are calculated. By comparing the position of the visual lane line, and utilizing the method of online calibration of camera calibration parameters, online calibration of the camera can be achieved, which can avoid camera perception failure caused by abnormal conditions and improve the accuracy of detection.

[0132] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A calibration method for calibrating cameras on unmanned trucks, characterized in that, Includes the following steps: Unmanned trucks acquire trajectory information of their driving routes; The positioning system records and saves the marking information of the unmanned truck when it travels according to the trajectory information, and converts the marking information into a coordinate system; The guardrails on the road section are identified by LiDAR, and a guardrail line is selected and fitted based on the point cloud data of the identified guardrails. The visual lane lines are obtained by acquiring the fitted lane lines on both sides of the driving road segment through a camera, and verifying the fitted lane lines on both sides based on the fitted guardrail lines; including: when acquiring images through the camera, acquiring the values ​​of the rotation vector pitch, yaw and roll, as well as the translation vector variable value Z; The height values ​​of the camera's rotation and translation vectors are iteratively corrected online. The pitch deviation angle is set to within 0.5°, and it is determined whether the slope of the visual lane line satisfies the first preset formula. If the condition is not met, the initial correction angle value AngStep is selected as 5°. If the first preset condition is met, the visual lane line formula formed by the conversion of the camera's intrinsic and extrinsic parameters is re-acquired and recalculated. The value of AngStep is half of the previous value each time it is re-corrected. If satisfied, no correction is needed or pitch correction is complete; Set the yaw deviation angle range to within 0.5°, and determine whether the slope of the visual lane line satisfies the second preset formula: If not satisfied, the initial correction angle value AngStep is selected as 5°. If the second preset condition is met, the visual lane line formula formed by the conversion of the camera's intrinsic and extrinsic parameters is re-acquired and recalculated. The value of AngStep is half of the previous value each time it is re-corrected. If satisfied, no correction is needed or yaw correction is completed. Set the roll deviation threshold to within 0.05m and determine whether the slope of the visual lane line meets the third preset formula; If not satisfied, the initial correction angle value AngStep is selected as 5°. If the third preset condition is met, the visual lane line formula formed by the conversion of the camera's intrinsic and extrinsic parameters is recalculated. The value of AngStep is half of the previous value each time it is recalibrated. If satisfied, no correction is required or the roll correction is completed. The lane width Wtrue is obtained by using the positioning information, and the correction threshold is set to 0.1 meters. The system then determines whether the width Wcamera of the unmanned truck and the lane width Wtrue satisfy the fourth preset formula. If not satisfied, select an initial correction height value of HighStep = 0.5; and if the fourth preset condition is met, re-acquire the visual lane line formula formed by the conversion of the camera's intrinsic and extrinsic parameters for calculation; the value of HighStep is half of the previous value each time it is re-corrected. If the conditions are met, no correction is needed or Z-correction is completed; obtain the corrected rotation vector values ​​and translation vector variable values; The visual lane lines are verified based on the corrected rotation vector values ​​and translation vector values ​​to obtain the corrected visual lane lines.

2. The calibration method for unmanned truck camera calibration according to claim 1, characterized in that: The coordinate system is either the UTM coordinate system or a latitude and longitude coordinate system.

3. The calibration method for unmanned truck camera calibration according to claim 2, characterized in that: The vehicle body coordinate system is obtained based on the coordinate system transformation, and the first straight line of the unmanned truck is fitted according to the least squares method. A threshold is manually set to filter point cloud data, and the distance between the filtered point cloud data and the first straight line is calculated. All point cloud data that are less than the mechanical threshold are collected to obtain a point set. The point set is proportionally divided into three segments: the first segment, the middle segment, and the last segment. A point cloud is selected from the point set of each segment to obtain the second straight line between the first segment and the middle segment or between the last segment and the middle segment, as well as the third straight line between the first segment and the last segment. The fourth line is obtained by calculating a weighted average of the second and third lines. Traverse the remaining point cloud within the point set and calculate the average distance from the remaining point cloud to the fourth line; Discard point clouds whose distance between the two endpoints of the fourth straight line is less than 30 meters, and calculate the ratio R of point clouds within the qualified range to point clouds in the point set. If the ratio R is greater than or equal to 0.8, perform at least 100 repeated iterations to obtain the fifth straight line of the final guardrail line; if the ratio R is less than 0.8, skip the current iteration. Based on the slope and intercept of the fifth line, Kalman tracking stability testing is performed to obtain the final expression for the laser guardrail line: y = k fence ×x+b fence ;k fence For the slope and b of the final laser guardrail line fence This is the intercept of the final laser guardrail line.

4. The calibration method for unmanned truck camera calibration according to claim 3, characterized in that: Obtain the angle between the heading of the unmanned truck and the first straight line, and perform manual threshold filtering of the guardrail point cloud when the angle is less than or equal to 15°.

5. The calibration method for unmanned truck camera calibration according to claim 3, characterized in that: If the ratio R is greater than 0.98, then the fourth straight line is directly determined as the fifth straight line of the final guardrail line.

6. The calibration method for unmanned truck camera calibration according to claim 1, characterized in that: The trajectory information is a straight road. The image information of the driving section is obtained by the camera, and the left lane line point set P1 and the right lane line point set P2 of the vehicle coordinate system are fitted by the UFLD model. Select any two point clouds in P1 and obtain the sixth straight line connecting the two point clouds. Calculate the distance from the remaining point cloud in P1 to the sixth line, and select the point cloud within 0.2 meters to obtain the candidate point cloud set, and obtain the average distance of the point cloud within the candidate point cloud set; Obtain the proportion M of the candidate point cloud set to all point clouds in P1. If the proportion M is greater than or equal to 0.9, perform multiple iterative calculations; if the proportion M is less than 0.9, recalculate by selecting any two point clouds in P1. After multiple iterations of calculation, the sixth straight line with the smallest average value is selected as the left lane line, and the slope and intercept values ​​are obtained by least squares method based on the cluster of candidate points. Similarly, calculate the seventh line of the P2 point set, as well as the slope and intercept of the seventh line, to obtain the right lane line; Based on the expression for the final laser guardrail line, the calculation formulas for the visual lane lines of the sixth and seventh straight lines are obtained:

7. The calibration method for unmanned truck camera calibration according to claim 6, characterized in that: If the ratio M is greater than or equal to 1, then the sixth straight line in this calculation is directly determined to be the left lane line.

8. The calibration method for unmanned truck camera calibration according to claim 6, characterized in that: When performing multiple iterations, the number of iterations shall not exceed 50, and the angle difference of the lane lines on the same side of the consecutive frames shall be calculated using the following formula: Where i is 6 and 7; Determine if the angle value is less than or equal to 2° for five consecutive times. If so, perform track tracking of the unmanned container truck; otherwise, recalculate.

9. The calibration method for unmanned truck camera calibration according to claim 1, characterized in that: Select x1 and x2 as adjustment points on the visual lane line, where x2-x1≥10 and x1≥0. The formula for calculating the width W of the unmanned truck is... camera : The first preset formula is: Where PI = π; The first preset condition is: and pitch cur =pitch pre -AngStep, or, and pitch cur =pitch pre +AngStep; The second preset formula is: The second preset condition is: And yaw cur =yaw pre -AngStep, or, And yaw cur =yaw pre +AngStep; The third preset formula is: The third preset condition is: And roll cur =roll pre -AngStep, or, And roll cur =roll pre +AngStep; The fourth preset formula is: The fourth preset condition is: and Z cur =Z pre -HightStep, or, and Z cur =Z pre +HightStep.

10. The calibration method for unmanned truck camera calibration according to claim 9, characterized in that: The values ​​of pitch, yaw, and roll of the rotation vector and the value of Z of the translation vector are corrected at least S times, and S≥100. The average value of pitch, yaw, roll, or Z obtained after S corrections is calculated as the final correction value. Each time the camera is calibrated and the final calibration value is obtained for the first time, it is counted as one round of calculation. After repeating the calculation for M rounds, the proportion T of the total number of camera calibrations that need to be performed is calculated. If T is greater than 0.2, then the statistical correction should be restarted. If T is less than or equal to 0.2, then the counting of round M is restarted, and the pitch value is assigned as a multiple of N, where 3 > N > 1.

Images