A vehicle-mounted infrared camera external parameter calibration method and system

Abstract

The present application relates to a kind of vehicle-mounted infrared camera external parameter calibration method and system, comprising the following steps: by being mounted on vehicle infrared camera obtains multiple infrared images and saves;According to the infrared image obtained by infrared camera, determine whether the vehicle body is parallel to lane line in the process of vehicle driving;According to single-frame infrared image, the pitch angle and yaw angle of the infrared camera when obtaining current frame infrared image are estimated;And the pitch angle and yaw angle when infrared camera obtains continuous frame infrared image are estimated, to complete the dynamic calibration of infrared camera external parameter.The present application method is simple to operate, and resource consumption is low, without using professional calibration site and calibration target, it only needs to keep the vehicle body parallel to lane line, then infrared image can be collected to carry out the dynamic calibration of infrared camera external parameter.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to a method and system for calibrating the extrinsic parameters of an in-vehicle infrared camera. Background Technology

[0002] Autonomous vehicles are widely used in various scenarios such as intelligent transportation, logistics and delivery, and cleaning operations. As the main sensor of autonomous vehicles, cameras are often used to detect the position of obstacles during vehicle operation. The accuracy of the detection results is directly related to the camera's extrinsic parameters, so it is necessary to calibrate the extrinsic parameters of the vehicle-mounted cameras.

[0003] Currently, camera extrinsic parameters are calibrated primarily through the following methods:

[0004] 1. A calibration site is set up manually, and a specific calibration board or calibration object is set up in the site. However, due to the high requirements for the relative positional accuracy between the vehicle and the calibration board / calibration object, the overall cost of this solution is high.

[0005] 2. Visible light cameras can be calibrated using methods such as checkerboard calibration. However, due to the characteristics of infrared images, such as low contrast, wide dynamic range, image discontinuity, low signal-to-noise ratio, and low texture, calibrating infrared cameras using the same methods as visible light cameras will result in problems such as low calibration accuracy. Summary of the Invention

[0006] The purpose of this invention is to provide a method and system for calibrating the extrinsic parameters of a vehicle-mounted infrared camera. The method is simple to operate, consumes little resources, and does not require a professional calibration site or calibration target. It only requires keeping the vehicle body parallel to the lane line and then acquiring infrared images to perform dynamic calibration of the infrared camera's extrinsic parameters.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] On the one hand, a method for calibrating the extrinsic parameters of a vehicle-mounted infrared camera is provided, which includes the following steps:

[0009] Multiple frames of infrared images are acquired and saved using an infrared camera mounted on the vehicle;

[0010] The infrared images obtained by the infrared camera determine whether the vehicle body is parallel to the lane lines during the vehicle's movement.

[0011] Based on a single frame of infrared image, the pitch and yaw angles of the infrared camera are estimated when the current frame of infrared image is acquired;

[0012] In addition, the pitch and yaw angles of the infrared camera are estimated when it acquires consecutive frames of infrared images, so as to complete the dynamic calibration of the infrared camera's extrinsic parameters.

[0013] On the other hand, a vehicle-mounted infrared camera extrinsic parameter calibration system is also provided, which can be used to implement the above-mentioned vehicle-mounted infrared camera extrinsic parameter calibration method. The vehicle-mounted infrared camera extrinsic parameter calibration system includes:

[0014] An image storage unit is used to store multiple frames of infrared images acquired by an infrared camera mounted on the vehicle during vehicle operation;

[0015] The parallel judgment unit is used to determine whether the vehicle body is parallel to the lane line during the vehicle's movement based on the infrared image acquired by the infrared camera.

[0016] The single-frame calibration unit is used to estimate the pitch and yaw angles of the infrared camera when the current frame of infrared image is obtained, based on a single frame of infrared image.

[0017] And a continuous frame calibration unit, which is used to obtain the pitch angle and yaw angle when the infrared camera acquires continuous frame infrared images through extended Kalman filtering and other methods.

[0018] In summary, the present invention has the following advantages compared with the prior art:

[0019] The vehicle-mounted infrared camera extrinsic parameter calibration method of this invention is simple to operate, consumes few resources, and requires no professional calibration site or calibration target. It only requires keeping the vehicle body parallel to the lane line and then acquiring infrared images to dynamically calibrate the infrared camera's extrinsic parameters. The specific process can be achieved using only lane line detection points. During the calibration process, detection points that deviate too far from the lane line are removed, and detection point elements with small deviations (i.e., effective detection elements) are selected to reduce errors in subsequent angle estimation and improve the accuracy of the estimation results. Attached Figure Description

[0020] Figure 1 This is a flowchart of the extrinsic parameter calibration method for a vehicle-mounted infrared camera disclosed in this invention.

[0021] Figure 2 This is a schematic diagram showing the positional relationship between the vehicle and the lane lines.

[0022] Figure 3 This is a schematic diagram of lane line detection points obtained through an algorithm.

[0023] Figure 4 This is a schematic diagram for fitting the left lane line and fitting the right lane line.

[0024] Figure 5 This is a schematic diagram showing the relationship between lines P1, P2, P3, P4 and the vanishing point VP1.

[0025] Figure 6 This is a schematic diagram of the external parameter calibration system for a vehicle-mounted infrared camera disclosed in this invention. Detailed Implementation

[0026] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0027] Example 1

[0028] like Figure 1 As shown, this embodiment provides a method for calibrating the extrinsic parameters of a vehicle-mounted infrared camera, which includes the following steps:

[0029] S1. Calibration condition initialization, which includes the following steps:

[0030] S11, such as Figure 2 As shown, a flat road surface with no obvious curves, clear and complete lane lines, and a length of ≥150m is selected as the calibration lane. There are two lane lines L, which are parallel to each other, and at least one of the two lane lines L is a solid line.

[0031] S12. Make the vehicle C equipped with infrared camera R enter the area between the two lane lines L, and make its body parallel to the lane line L.

[0032] S13. Start the vehicle, make vehicle C start moving, and adjust the vehicle speed v. When the vehicle speed v is greater than or equal to a predetermined value (e.g., 30km / h), the infrared camera R starts to acquire infrared images and acquires the vehicle speed v and vehicle steering angle θ in real time, thereby obtaining multiple frames of infrared images and saving them. Each frame of infrared image corresponds to a vehicle speed v and a vehicle steering angle θ.

[0033] Furthermore, the initialization of the calibration conditions also includes calibrating the intrinsic parameters of the infrared camera R to obtain the intrinsic parameter matrix K and distortion vector distcoff of the infrared camera R:

[0034]

[0035] distcoff = [k1 k2 p1 p2 k3]

[0036] in, f is the focal length of the infrared camera, dx and dy are the pixel size, and c x and c y The principal imaging point of the infrared camera is k1, k2, and k3, which are radial distortion coefficients, and p1 and p2 are tangential distortion coefficients.

[0037] In this embodiment, the intrinsic parameter matrix K and distortion vector distcoff of the infrared camera can be obtained through existing technologies, such as the publication number CN113989386A and the patent title "An Infrared Camera Calibration Method and System", which will not be described in detail here.

[0038] S2. Determine whether the vehicle body is parallel to the lane lines during vehicle movement based on the infrared images acquired by the infrared camera, which includes the following steps:

[0039] S21. Select an image set S from the saved multi-frame infrared images according to the filtering conditions, and the image set S contains N consecutive frame infrared images; the filtering conditions include: when the infrared image is acquired, the vehicle speed v is greater than or equal to the vehicle speed setting value (e.g., 30km / h), and when the infrared image is acquired, the vehicle steering angle θ is less than or equal to the steering angle setting value (e.g., 1°).

[0040] S22. If N is greater than or equal to a preset value (e.g., 10), then based on algorithms such as UFLDv2 (Ultra Fast Deep Lane Detection with Hybrid Anchor Driven Ordinal Classification), the lane detection points (e.g., ...) of each frame of infrared image in the image set S are obtained. Figure 3 As shown, it includes the coordinates of several left lane detection points P1 and several right lane detection points Pr) to obtain the set of left lane detection points {left} for each frame of infrared image. ij} and the set of right lane detection points {right ik}, where 1≤i≤N, 1≤j≤m, 1≤k≤n, m is the maximum number of left lane detection points in all infrared images, n is the maximum number of right lane detection points in all infrared images, left ij Let be the coordinates of the j-th detection point of the left lane line in the i-th infrared image of the image set S. ik Let K be the coordinates of the k-th detection point on the right lane line in the i-th image of the image set S.

[0041] It should be noted that if N is less than the preset value, it means that there are few infrared images that meet the screening conditions and cannot be used for the statistics and subsequent analysis of lane line detection points. Therefore, return to step S1 and obtain infrared images again until N is greater than or equal to the preset value.

[0042] S23, such as Figure 4 As shown, based on the set of left lane line detection points {left ij The fitting of the left lane line y = k for each frame of the infrared image in the image set S is obtained by fitting the image. li x+b lAnd based on the right lane detection point {rightik}, the fitted right lane line y=k of each frame of infrared image in image set S is obtained. ri x+b r ; where k li b l Let k be the slope and intercept of the fitted left lane line of the i-th infrared image in the image set S. ri b r Let be the slope and intercept of the fitted right lane line of the i-th infrared image in image set S, respectively.

[0043] S24. Let x = 0, and according to the fitted left lane line y = k li x+b l Obtain the set of intersection points {y} between the fitted left lane line and the boundary of the infrared image. i}; or, let x = 0, and according to the fitted right lane line y = k ri x+b r Obtain the set of intersection points {y} between the fitted right lane line and the boundary of the infrared image. i};

[0044] If the set of intersection points {y i If the preset conditions are met, the current vehicle body is considered to be parallel to lane line L, and step S3 is executed; otherwise, step S1 is returned. Further, in this embodiment, the preset conditions are: the set of intersection points {y...} i In the formula, the maximum value minus the minimum value is less than or equal to Y, and 50 is less than or equal to Y and 200, and the values ​​are positive integers, such as Y = 120 in this embodiment;

[0045] For example, such as Figure 4 As shown, the ordinate of the intersection point of the fitted left lane line and the left boundary of the infrared image is denoted as Y, and the origin of the coordinate system is set at the upper left corner of the infrared image. Similarly, the ordinate of the intersection point of the fitted right lane line and the right boundary of the infrared image is also denoted as Y, and the origin of the coordinate system is set at the upper right corner of the infrared image.

[0046] Therefore, since this invention uses the lane line direction as the z-axis of the world coordinate system and the plane perpendicular to the z-axis as the xoy plane, and performs pitch and yaw estimation and correction on this basis, it is necessary to ensure that the vehicle body is parallel to the lane line L. The above technical solution can quickly determine whether the vehicle body is parallel to the lane line L, so as to avoid excessive errors in the subsequent yaw calculation and ensure the accuracy of the results.

[0047] S3. Based on a single infrared image from N consecutive frames of image set S, estimate the pitch and yaw angles of the infrared camera when acquiring the current frame of infrared image. This includes the following steps:

[0048] S31. Based on the N consecutive frames of the image set S, the set of left lane detection points in the current frame infrared image {left} ij Construct a subset of left lane line detection points, wherein the subset of left lane line detection points = {(left i1 left i2 L1), (left) i1 left i3 L2), ...(left) iX1-1 left X1 LT l The subset of left lane detection points contains X1 left lane detection points (i.e., left lane detection points). i1 left i2 left i3 ...left iX1-1 left X1 ) and T l Each left lane line detection point element (i.e., (left lane line detection point element) i1 left i2 L1), (left) i1 left i3 L2), ...(left) iX1-1 left X1 LT l Each left lane detection point element contains a subset of the left lane detection points {left}. ij Two detection points are randomly selected from the data, and the distance between these two detection points (i.e., L1, L2...LT) is used as the reference. l ), and the subset of left lane line detection points contains the set of left lane line detection points {left ij The coordinates of each detection point in}, and the coordinates of the two detection points in each left lane line detection point element are different;

[0049] And determine the maximum value L_max1 (i.e., L1, L2...LT) of the distance between any two detection points in all left lane line detection point elements. l (the maximum value in the middle);

[0050] For example, suppose the set of left lane line detection points is {left ij There are 3 detection point coordinates in}, left 11 left 12 left 13 From the set of left lane line detection points {left ij In each iteration, two detection points are randomly selected, such as left. 11 left 12And calculate the distance L1 between the two, then (left) 11 left 12 L1) is used as a left lane detection point element, and this process is repeated several times until the set of left lane detection points {left} is reached. ij The coordinates of each detection point in the array are obtained, and then duplicate left lane line detection point elements are removed, such as (left 11 left 11 L3), (left) 12 left 12 L4), (left) 13 left 13 L5), the remaining left lane line detection point elements, i.e. (left 11 left 12 、L1)(left 11 left 13 L2), (left) 12 left 13 L3) is constructed as a subset of left lane line detection points, and this subset of left lane line detection points contains 3 left lane line detection point elements;

[0051] Then, determine the maximum value from the distance values ​​L1, L2, and L3 between the two detection points of each left lane line detection point element, and denot it as L_max1;

[0052] Similarly, based on the set of right lane detection points in the current frame of the infrared image {right ik Construct a subset of right lane detection points, wherein the set of right and left lane detection points = {(right i1 right i2 R1), (right) i1 right i3 R2), ...(right) iX2-1 right X2 RT r The subset of right lane detection points contains X2 right lane detection points (i.e., right lane detection points). i1 right i2 right i1 right i3 ...right iX2-1 right X2 ) and T r Each right lane line detection point element (i.e., right) i1 right i2 R1), (right) i1 righti3 R2), ...(right) iX2-1 right X2 RT r Each right lane detection point element contains a set of right lane detection points. ik Two detection points are randomly selected from the data, and the distance between these two detection points (i.e., R1, R2...RT) is used as the reference. r ), and the right lane detection point subset contains the right lane detection point set {right ik The coordinates of each detection point in}, and the coordinates of the two detection points in each right lane line detection point element are different;

[0053] And determine the maximum value L_max2 (i.e., R1, R2...RT) of the distance between any two detection points in the right lane line detection points. r (the maximum value in the middle);

[0054] Finally, the set of left lane detection points and the set of right lane detection points are combined to obtain the set of lane detection points in the current image.

[0055] S32, such as Figure 5 As shown, arbitrarily select a left lane line detection point element from the subset of left lane line detection points (e.g., (left lane line detection point)). 11 left 12 L1)) and obtain the straight line P1 passing through two detection points in the detection point element, and arbitrarily select one right or left lane detection point element from the right lane detection point subset (such as (right) 11 right 12 R1)) and obtain the straight line P2 passing through the two detection points in the detection point element;

[0056] Find the intersection point of lines P1 and P2, and denote this intersection point as the lane line vanishing point VP1;

[0057] S33. Traverse the detection point elements in the current image lane line detection point subset to obtain the deviation degree of each detection point element from the lane line, and save the valid detection point elements whose deviation degree meets the preset conditions and the sum of the deviation values ​​of the valid detection point elements.

[0058] Specifically, it includes the following steps:

[0059] S331. Obtain the straight line P3 passing through the two detection points in the current detection point element, and the straight line P4 connecting any point (preferably the midpoint of the two detection points) on the line connecting the two detection points in the current detection point element and the vanishing point VP1 of the lane line.

[0060] For example, the current detection point element is (left) 31 left 22 L5), determine the two detection points left among them. 31 left 22 The straight line P3 and the midpoint left of the two detection points mid Then determine the path through the midpoint left. mid The line P4 intersects the vanishing point VP1;

[0061] S332. Obtain the acute angle α formed by lines P3 and P4; in this embodiment, the angle threshold value ranges from [0.2°-2.0°], preferably from [0.5°-1.5°], and particularly preferably from [0.5°-1.0°].

[0062] If the acute angle α is greater than the angle threshold, it means that the current detection point element deviates far from the lane line and should not be used in subsequent analysis steps.

[0063] If the acute angle α is less than or equal to the angle threshold, it indicates that the current detection point element deviates from the lane line by a small degree (i.e., the degree of deviation meets the preset conditions), and the current detection point element is a valid detection point element. The deviation value (score) of the current detection point element relative to the lane line is further calculated using formula (1):

[0064]

[0065] Where L is the distance between two detection points in the current detection point element in the current lane line detection point subset of the current image; when the current detection point element comes from the left lane line detection point subset, L_max is L_max1, and when the current detection point element comes from the right lane line detection point subset, L_max is L_max2.

[0066] S333. Repeat steps S331-S332 to traverse each detection point element in the current image lane line detection point subset, and sum all the obtained deviation values ​​score. The summation result is recorded as the total deviation value score1 and saved. At the same time, the effective detection point element set Lcc is used to save all detection point elements whose included angle α is less than or equal to the angle threshold.

[0067] Therefore, through the above steps, detection points that deviate far from the lane line can be quickly removed, and detection point elements with small deviations (i.e., effective point detection elements) can be selected to reduce the error in subsequent angle estimation and improve the accuracy of the estimation results. For each frame of infrared image, there are several vanishing points VP1, and each vanishing point VP1 corresponds to a score1 value and a set of effective detection point elements Lcc.

[0068] S34. Repeat steps S32-S33 until the sum of several deviation values, score1, of the current frame infrared image in N consecutive frames of image set S is obtained, and construct the set of deviation value sums {score1}, and determine the maximum value score1 in the set of deviation value sums {score1}. max And save the maximum value score1 max The corresponding vanishing point VP1 is denoted as vanishing point Svp, and the corresponding set of valid detection point elements Lcc is denoted as Lc. Thus, for each frame of infrared image, it has 1 vanishing point Svp and 1 set Lc.

[0069] S35. Based on the intrinsic parameter matrix K of the infrared camera and the maximum value score1 max The corresponding set of valid detection points Lc is used to calculate the pitch and yaw angles of the infrared camera when acquiring the current frame of infrared image. This process includes the following steps:

[0070] S351. Based on the transformation equation from the image coordinate system to the camera coordinate system, calculate the maximum value score1 using the intrinsic parameter matrix K. max In the corresponding set of valid detection point elements Lc, the unit spherical coordinates of the two detection points in each detection point element, and the maximum value score1 calculated according to formula (2). max In the corresponding set of valid detection points Lc, the two detection points in each detection point element form the cross set vector n of the vector:

[0071] n = (K -1 p1)×(K -1 p2) (2)

[0072] S352. Construct matrix A using all cross set vectors. To solve for Av = 0, perform SVD decomposition on matrix A to obtain the vanishing point VP1 direction vector v' (which is the optimal solution).

[0073] S353. Project the vanishing point VP1 direction vector v' onto the imaging plane according to formula (3) to obtain the optimal vanishing point v. p The optimal vanishing point v p That is, the optimal vanishing point obtained by solving for all valid detection point elements;

[0074] v p =Kv' (3)

[0075] S354. Assuming the roll angle is 0, the pitch and yaw angles of the infrared camera R when acquiring the current frame infrared image are obtained according to formula (4).

[0076]

[0077] Where (u0 v0) is the optimal vanishing point v p The coordinates (XYZ) represent the optimal vanishing point v. p The point corresponding to the world coordinate system, and since the two lane lines L will not intersect in the world coordinate system, Z = +∞; λ is the scale factor; R is the rotation matrix between the world coordinate system and the camera coordinate system; T is the translation matrix between the world coordinate system and the camera coordinate system.

[0078] S4. Repeat step S3 to estimate the pitch and yaw angles corresponding to each single frame of infrared image in the image set S obtained by the infrared camera.

[0079] S5. Simulate the changes in pitch and yaw angles of the infrared camera during vehicle movement using methods such as a constant angular velocity model. Then, use extended Kalman filtering and other methods to estimate the pitch and yaw angles when the infrared camera acquires continuous frames of infrared images, so as to complete the dynamic calibration of the infrared camera's extrinsic parameters.

[0080] Therefore, the vehicle-mounted infrared camera extrinsic parameter calibration method in this embodiment is simple to operate, consumes few resources, and does not require a professional calibration site or calibration target. It only requires keeping the vehicle body parallel to the lane line and then acquiring infrared images to perform dynamic calibration of the infrared camera extrinsic parameters. In the specific calibration process, detection points that deviate far from the lane line are removed, and detection point elements with small deviations (i.e., effective point detection elements) are selected to reduce the error in subsequent angle estimation and improve the accuracy of the estimation results.

[0081] Example 2:

[0082] This embodiment provides a vehicle-mounted infrared camera extrinsic parameter calibration system, which can implement the vehicle-mounted infrared camera extrinsic parameter calibration method in Embodiment 1, such as... Figure 6 As shown, the vehicle-mounted infrared camera extrinsic parameter calibration system includes:

[0083] Image storage unit 1 is used to store multiple frames of infrared images acquired by an infrared camera mounted on the vehicle during vehicle operation. The multiple frames of infrared images are acquired when the vehicle speed v is greater than or equal to a predetermined value, and each frame of infrared image corresponds to a vehicle speed v and a vehicle steering angle θ.

[0084] Parallel judgment unit 2 is used to determine whether the vehicle body is parallel to the lane line during the vehicle's driving process based on the infrared image acquired by the infrared camera. The specific process is described in step S2 of the embodiment and will not be repeated here.

[0085] The single-frame calibration unit 3 is used to estimate the pitch and yaw angles of the infrared camera when the current frame infrared image is obtained based on the single-frame infrared image. The specific process is described in steps S3-S4 of the embodiment and will not be repeated here.

[0086] And a continuous frame calibration unit 4, which is used to obtain the pitch and yaw angles when the infrared camera acquires continuous frame infrared images through extended Kalman filtering and other methods.

[0087] In summary, the vehicle-mounted infrared camera extrinsic parameter calibration method of this invention is simple to operate, consumes few resources, and requires no professional calibration site or calibration target. It only requires keeping the vehicle body parallel to the lane line and then acquiring infrared images to dynamically calibrate the infrared camera's extrinsic parameters. The specific process can be achieved using only lane line detection points. Furthermore, during the calibration process, detection points that deviate too far from the lane line are removed, and detection point elements with minimal deviation (i.e., effective detection elements) are selected to reduce errors in subsequent angle estimation and improve the accuracy of the estimation results.

[0088] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for calibrating the extrinsic parameters of a vehicle-mounted infrared camera, characterized in that, Includes the following steps: Multiple frames of infrared images are acquired by an infrared camera mounted on the vehicle; The infrared images obtained by the infrared camera determine whether the vehicle body is parallel to the lane lines during the vehicle's movement. When the vehicle body is parallel to the lane line, the pitch angle and yaw angle of the infrared camera are estimated when a single frame of infrared image is acquired. And to estimate the pitch and yaw angles when the infrared camera acquires consecutive frames of infrared images, so as to complete the dynamic calibration of the infrared camera's extrinsic parameters; The process of estimating the pitch and yaw angles of the infrared camera based on a single frame of infrared image includes the following steps: Construct subsets of detection points for the left and right lane lines in the current frame of the infrared image; The left and right lane line detection point sets are combined to obtain the current image lane line detection point set; Iterate through the detection point elements in the current image lane line detection point subset to obtain the degree of deviation of each detection point element from the lane line, and save the valid detection point elements whose deviation degree meets the preset conditions and the sum of the deviation values ​​of the valid detection point elements. Based on the intrinsic parameter matrix of the infrared camera and the set of effective detection point elements corresponding to the maximum value of the sum of deviation values, the pitch angle and yaw angle of the infrared camera when acquiring the current frame of infrared image are calculated. The deviation value (score) of the current detection point element relative to the lane line is calculated using formula (1): ； Where L is the distance between two detection points in the current detection point element within the current lane line detection point subset; when the current detection point element comes from the left lane line detection point subset, L_max is the maximum distance between two detection points in all left lane line detection point elements; when the current detection point element comes from the right lane line detection point subset, L_max is the maximum distance between two detection points in all right lane line detection point elements; α is the acute angle formed by the straight line passing through two detection points in the current detection point element and the straight line formed by any point on the line connecting two detection points in the current detection point element and the vanishing point of the lane line.

2. The method for calibrating the extrinsic parameters of a vehicle-mounted infrared camera as described in claim 1, characterized in that, Determining whether a vehicle is parallel to the lane lines during its journey based on infrared images acquired by an infrared camera includes the following steps: Obtain the coordinates of lane line detection points in the infrared image to obtain the lane line detection points for each frame of the infrared image; The fitted lane line is obtained based on the set of lane line detection points; Obtain the set of intersection points between the fitted lane line and the boundary of the infrared image; If the set of intersection points meets the preset conditions, then the current vehicle body is considered to be parallel to the lane line.

3. The method for calibrating the extrinsic parameters of a vehicle-mounted infrared camera as described in claim 2, characterized in that, When acquiring each frame of infrared image, the vehicle speed is greater than or equal to the vehicle speed setting value, and the vehicle steering angle is less than or equal to the steering angle setting value.

4. The method for calibrating the extrinsic parameters of a vehicle-mounted infrared camera as described in claim 2, characterized in that, The preset condition is: in the set of intersection points, the maximum value minus the minimum value ≤ Y, and 50 ≤ Y ≤ 200.

5. The method for calibrating the extrinsic parameters of a vehicle-mounted infrared camera as described in claim 1, characterized in that, Traverse the detection point elements in the current image's lane line detection point subset to obtain the deviation degree of each detection point element from the lane line, and save the valid detection point elements whose deviation degree meets the preset conditions and the sum of the deviation values ​​of the valid detection point elements, including the following steps: Obtain the straight line P3 passing through the two detection points in the current detection point element, and the straight line P4 connecting any point on the line connecting the two detection points in the current detection point element and the vanishing point VP1 of the lane line; Obtain the angle α formed by lines P3 and P4; If the acute angle α is less than or equal to the angle threshold, then the current detection point element is a valid detection point element, and the deviation value of the current detection point element relative to the lane line is calculated. Iterate through each detection point element in the current image's lane line detection point subset, sum all the obtained deviation values, record the sum as the total deviation value, and save it.

6. The method for calibrating the extrinsic parameters of a vehicle-mounted infrared camera as described in claim 5, characterized in that, The process of obtaining the lane vanishing point VP1 includes: obtaining a straight line P1 that passes through two detection points in any detection point element of the left lane detection point subset, and obtaining a straight line P2 that passes through two detection points in any detection point element of the right lane detection point subset, and recording the intersection of straight lines P1 and P2 as the lane vanishing point VP1.

7. The method for calibrating the extrinsic parameters of a vehicle-mounted infrared camera as described in claim 5, characterized in that, Based on the intrinsic parameter matrix of the infrared camera and the set of valid detection points corresponding to the maximum value of the sum of deviations, the pitch and yaw angles of the infrared camera are calculated when acquiring the current frame of the infrared image, including the following steps: In the set of valid detection point elements corresponding to the maximum sum of deviation values, the two detection points in each detection point element form the cross set vector of the vector; By constructing matrix A using all the cross set vectors, and solving matrix A, we can obtain the direction vector of the lane vanishing point VP1. v’ ; The direction vector of the lane line vanishing point VP1 v’ Project onto the imaging plane to obtain the optimal vanishing point. v p ; Based on the optimal vanishing point v p The coordinates of the infrared camera and their corresponding points in the world coordinate system are used to obtain the pitch and yaw angles of the infrared camera when acquiring the current frame of the infrared image.

8. The method for calibrating the extrinsic parameters of a vehicle-mounted infrared camera as described in claim 1, characterized in that, Estimating the pitch and yaw angles of the infrared camera when acquiring consecutive frames of infrared images is crucial for the dynamic calibration of the infrared camera's extrinsic parameters. This involves the following steps: The changes in the pitch and yaw angles of the infrared camera during vehicle movement are simulated, and then the pitch and yaw angles of the infrared camera when acquiring consecutive frames of infrared images are estimated using an extended Kalman filter, so as to complete the dynamic calibration of the infrared camera's extrinsic parameters.

9. A vehicle-mounted infrared camera extrinsic parameter calibration system, which can be used to implement the vehicle-mounted infrared camera extrinsic parameter calibration method according to any one of claims 1-8, characterized in that, The vehicle-mounted infrared camera extrinsic parameter calibration system includes: An image storage unit is used to store multiple frames of infrared images acquired by an infrared camera mounted on the vehicle during vehicle operation; The parallel judgment unit is used to determine whether the vehicle body is parallel to the lane line during the vehicle's movement based on the infrared image acquired by the infrared camera. The single-frame calibration unit is used to estimate the pitch and yaw angles of the infrared camera when the current frame of infrared image is obtained, based on a single frame of infrared image. And a continuous frame calibration unit, which is used to obtain the pitch angle and yaw angle when the infrared camera acquires continuous frame infrared images through extended Kalman filtering and other methods.

Images