A method, medium and system for correcting camera extrinsic parameters for road surface detection

Abstract

The application discloses a camera external parameter correction method, medium and system for road surface detection, comprising the following steps: taking the road surface starting point as the origin, the horizontal component of the vehicle moving mileage as the horizontal coordinate, and the road surface longitudinal profile elevation as the vertical coordinate, a broken line is drawn; obtaining the first coordinate of the starting point and the second coordinate of the ending point of the straight line segment where the vehicle is located in the broken line graph, the third coordinate of the ending point of the straight line segment where the intersection of the camera optical axis corresponding to the camera located point and the broken line of the broken line graph is located; calculating the coordinates of the vehicle located point; calculating the coordinates of the camera located point; establishing the first straight line equation corresponding to the camera optical axis in the broken line graph, and the second straight line equation corresponding to the straight line segment where the intersection of the camera optical axis corresponding to the camera located point and the broken line of the broken line graph is located; solving the slope of the first straight line equation, the slope and the intercept of the second straight line equation; calculating the corrected pitch angle of the camera and the corrected installation height of the camera. The application does not need to depend on the road surface reference object, and has better applicability.

Description

Technical Field

[0001] This invention relates to the field of camera extrinsic parameter calibration technology, and in particular to a method, medium, and system for correcting camera extrinsic parameters for road surface detection. Background Technology

[0002] In automated pavement distress detection, images of the pavement are captured using an area scan camera, and the distresses within these images are then identified. When using an area scan camera for pavement detection, camera parameters are needed to perform coordinate transformation and calculate the actual physical dimensions of the distresses. Therefore, accurate intrinsic and extrinsic parameters are required. Intrinsic parameters are internally set parameters of the camera and remain unchanged after being set. However, extrinsic parameters determine the camera's position and orientation. Ideally, the relative pose of the camera and the pavement is fixed, meaning the extrinsic parameters of the camera relative to the pavement are fixed, and the actual physical dimensions of the distresses can be calculated using monocular camera ranging technology. However, in actual automated pavement distress detection, the complexity of the road environment, the unevenness of the pavement surface, and the swaying of the camera carrier cause the relative pose of the camera and the pavement to deviate from a fixed position, altering the camera's extrinsic parameters. This leads to two problems: firstly, describing the camera photogrammetry system model with fixed extrinsic parameters without correction results in significant errors under actual road conditions, affecting coordinate transformation and the calculation of the actual physical dimensions of the distresses; secondly, error correction techniques based on known geometric features such as pavement markings rely entirely on pavement references, limiting their applicability. Summary of the Invention

[0003] This invention provides a method, medium, and system for correcting camera extrinsic parameters for road surface detection, in order to solve the problem that existing methods for calculating camera extrinsic parameters rely on road surface references, resulting in limited applicability.

[0004] Firstly, a method for correcting camera extrinsic parameters for road surface detection is provided, including:

[0005] While the vehicle is traveling on the road, the camera's optical axis is used to collect data in real time, including the angle between the camera's optical axis and the horizontal direction of the ground, the elevation of the road's longitudinal profile, and the distance the vehicle travels.

[0006] With the starting point of the road surface as the origin, the horizontal direction as the horizontal axis, the vertical direction as the vertical axis, the horizontal component of the vehicle's travel distance as the horizontal coordinate, and the elevation of the road surface longitudinal section as the vertical coordinate, a broken line is drawn to obtain a broken line graph.

[0007] When the vehicle travels to a certain position, the first coordinate of the starting point and the second coordinate of the ending point of the straight line segment where the vehicle is located in the polyline graph are obtained, as well as the third coordinate of the ending point of the straight line segment where the intersection of the camera optical axis corresponding to the camera location and the polyline of the polyline graph is located.

[0008] Based on the first coordinate, the second coordinate, and the vehicle's travel mileage, calculate the coordinates of the point where the vehicle is located in the line graph;

[0009] The coordinates of the camera in the polyline graph are calculated based on the first coordinate, the second coordinate, the coordinates of the vehicle location, and the camera installation height.

[0010] Establish the first straight line equation corresponding to the camera optical axis in the polyline graph, and the second straight line equation corresponding to the straight line segment where the intersection of the camera optical axis at the point where the camera is located and the polyline of the polyline graph is located.

[0011] The slope of the first straight line equation is determined based on the angle between the camera and the horizontal direction of the ground, and the slope and intercept of the second straight line equation are determined based on the second coordinate and the third coordinate.

[0012] The corrected pitch angle of the camera is calculated based on the slope of the first line equation and the slope of the second line equation, and the corrected mounting height of the camera is calculated based on the slope of the second line equation, the intercept of the second line equation, and the coordinates of the point where the camera is located.

[0013] In a second aspect, a computer-readable storage medium is provided, wherein computer program instructions are stored thereon; when executed by a processor, the computer program instructions implement the method for correcting camera extrinsic parameters for road surface detection as described in the first aspect embodiment above.

[0014] Thirdly, a system for calculating camera extrinsic parameters for road surface detection is provided, comprising: a computer-readable storage medium as described in the second aspect embodiment above.

[0015] Thus, in this embodiment of the invention, by collecting information on road longitudinal profile elevation, vehicle travel distance, and equipment attitude, the errors existing in the fixed external parameter description camera photogrammetry system model in practical applications are corrected, without relying on road reference objects, and thus have better applicability. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart of a camera extrinsic parameter correction method for road surface detection according to an embodiment of the present invention;

[0018] Figure 2 This is a schematic diagram of the camera's extrinsic parameters relative to the horizontal plane;

[0019] Figure 3 This is a line graph drawn according to an embodiment of the present invention;

[0020] Figure 4 This is a schematic diagram showing the positions of the vehicle and camera in a line graph according to an embodiment of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] This invention discloses a method for correcting the extrinsic parameters of a camera used for road surface detection. The camera described in this invention is generally an area array camera. The camera is typically mounted on the top of a vehicle and fixed with a rigid structure, ensuring that its optical axis is perpendicular to the road surface. Figure 2 The determined included angle γ (i.e., the camera's pitch angle) and the determined camera mounting height h are shown. The camera pitch angle γ and the camera mounting height h represent the camera's extrinsic parameters, which are the targets that need to be dynamically obtained in this embodiment of the invention.

[0023] by Figure 4 Taking the example shown, let's take a point H on the elevation break line of the road surface as the location of the vehicle. The line IH connecting camera I and the vehicle location point H is ideally perpendicular to road segment AB, which is the camera's installation height h. At this point, it can be observed that the intersection point J of the camera's optical axis and the ground is not on the same plane as AB. If calculated based on the original camera installation height h and pitch angle γ, it will lead to errors in the calculated points in the road surface coordinate system.

[0024] Therefore, it is necessary to calculate the camera's extrinsic parameters on the road surface plane where point J of the image is captured, namely the corrected pitch angle γ′ and the corrected mounting height h′. The vertical distance from camera I to the road surface plane BC, where the intersection of the camera's optical axis and the road surface is located, is the camera's mounting height at this point. Draw a perpendicular line from camera I to BC, intersecting the extension of BC at point K. At this point, IK is the corrected mounting height h′. The angle between the camera's optical axis and the perpendicular line of the road surface BC is the corrected pitch angle γ′, i.e., ∠KIJ is the pitch angle γ′. It should be understood that this diagram is for illustrative purposes only; the actual point J may lie on a line segment after BC.

[0025] Based on the above analysis, such as Figure 1 As shown, the method includes the following steps:

[0026] Step S101: While the vehicle is traveling on the road, the angle between the camera's optical axis and the horizontal direction of the ground, the elevation of the road longitudinal section, and the distance traveled by the vehicle are collected in real time.

[0027] The angle between the camera's optical axis and the horizontal direction of the ground is acquired using an inertial navigation system. This inertial navigation system is mounted at the same location on the vehicle and at the same angle as the camera, allowing for real-time acquisition of the camera's attitude information.

[0028] The elevation of the road surface longitudinal profile is collected by a longitudinal profile elevation detection device.

[0029] The vehicle's travel distance is collected by a ranging encoder.

[0030] During data collection, the angle between the camera's optical axis and the horizontal direction of the ground, as well as the elevation of the road surface longitudinal profile, are collected at fixed moving distance intervals. These two parameters are then correlated with the vehicle's moving mileage.

[0031] Step S102: Using the starting point of the road surface as the origin, the horizontal direction as the horizontal axis, the vertical direction as the vertical axis, the horizontal component of the vehicle's travel distance as the horizontal coordinate, and the elevation of the road surface longitudinal section as the vertical coordinate, draw a broken line to obtain a broken line graph.

[0032] For example, a line chart Figure 3 As shown, the broken line is formed by connecting multiple straight line segments in sequence.

[0033] Step S103: When the vehicle travels to a position, obtain the first coordinate of the starting point and the second coordinate of the ending point of the straight line segment where the vehicle is located in the polyline graph, as well as the third coordinate of the ending point of the straight line segment where the intersection of the camera optical axis and the polyline graph is located.

[0034] The ordinate of the endpoint of each straight line segment in the broken line graph is the elevation of its respective road surface longitudinal profile. Therefore, the ordinates of the first, second, and third coordinates are the elevations of their respective road surface longitudinal profiles.

[0035] The starting point of the road surface is the origin; therefore, the coordinates of the starting point are (0, 0). A line graph can be viewed as a series of consecutively connected straight line segments. Therefore, starting from the first straight line segment, we can calculate the x-coordinate of the endpoint of each segment sequentially, treating each segment as the hypotenuse of a right triangle, with one leg parallel to the x-axis and the other perpendicular to the x-axis. The general formula for this calculation is:

[0036]

[0037] Where (x1, y1) represents the coordinates of the starting point of a straight line segment in the broken line graph, (x2, y2) represents the coordinates of the ending point of a straight line segment in the broken line graph, and d represents the distance between the starting point and the ending point of the straight line segment, which is equal to the difference between the vehicle travel distance corresponding to the ending point of the straight line segment and the vehicle travel distance corresponding to the starting point of the straight line segment.

[0038] The starting point of the first straight segment is (0, 0), and the ordinate of its ending point is the collected road longitudinal profile elevation. Therefore, the abscissa of the ending point of the first straight segment can be calculated using the above formula. When calculating the second straight segment, the ending point of the first straight segment becomes the starting point of the second straight segment, and the ordinate of its ending point is the collected road longitudinal profile elevation. The abscissa of the ending point of the second straight segment is calculated using the above formula. This process continues until the starting and ending coordinates of each straight segment are obtained.

[0039] Therefore, the x-coordinates of the first, second, and third coordinates can be obtained using the above formula.

[0040] Step S104: Calculate the coordinates of the point where the vehicle is located in the line graph based on the first coordinate, the second coordinate, and the vehicle's travel distance.

[0041] In practical spatial geometric calculations, it's difficult to determine the direction vector of a straight line. Generally, the starting point O and ending point E of the line segment are known, so the direction vector is D = EO. Then, according to the vector equation of a ray, a point P on the line segment is: P = O + tD. The value of t determines the position of P on the line segment. When the direction vector is determined by the starting and ending points, and the point lies within the line segment, the range of t is 0 to 1: a value of 0 indicates the starting point O, and a value of 1 indicates the ending point E. Furthermore, according to the principle of similar triangles, if the distance d between point P and the starting point O is known, then the value of t is: Mod(D) is the magnitude of the vector, which is the length of the line segment.

[0042] by Figure 4 For example, given the coordinates of points A and B, and the length of AH (which is the absolute value s of the difference in the distance traveled by the vehicle between points A and B), the vector AB can be represented as (x... b -x a ,y b -y a If ), then the length of vector AB is Based on the above principles, the coordinates of the vehicle's location are calculated, and the results are obtained. The formula for calculating the coordinates of the vehicle's location is:

[0043]

[0044] Among them, (x a ,y a(x) represents the first coordinate. b ,y b (x) represents the second coordinate. h ,y h ) represents the coordinates of the vehicle's location. s represents the absolute value of the difference between the vehicle's location and the starting point of the straight line segment on which the vehicle's location lies.

[0045] Step S105: Calculate the coordinates of the camera's location in the polyline graph based on the first coordinate, the second coordinate, the coordinates of the vehicle's location, and the camera's installation height.

[0046] by Figure 4 For example, rotating vector AB counterclockwise by 90° yields the vector (y) in the HI direction. a -y b ,x b -x a Similarly, based on the aforementioned principles of spatial geometry, the coordinates of point I can be calculated to obtain... The formula for calculating the coordinates of the camera's location is:

[0047]

[0048] Among them, (x h ,y h (x) represents the coordinates of the location of the vehicle. i ,y i The coordinates () represent the location of the camera. The height (h) represents the camera's mounting height. This mounting height is generally determined based on the height of the vehicle (i.e., the vertical distance between the top of the vehicle and the road surface).

[0049] Step S106: Establish the equation of the first straight line corresponding to the camera optical axis in the polyline graph, and the equation of the second straight line corresponding to the line segment where the intersection of the camera optical axis and the polyline graph is located.

[0050] Specifically, the equation of the first straight line is:

[0051] y = a1x + b1.

[0052] Where a1 represents the slope of the first line equation, and b1 represents the intercept of the first line equation.

[0053] Specifically, the equation of the second line is:

[0054] y = a²x + b².

[0055] Where a2 represents the slope of the second line equation, and b2 represents the intercept of the second line equation.

[0056] Step S107: Solve for the slope of the first straight line equation based on the angle between the camera and the horizontal direction of the ground, and solve for the slope and intercept of the second straight line equation based on the second and third coordinates.

[0057] Specifically, the formula for calculating the slope of the first straight line equation is as follows:

[0058] a1 = -tanγ0.

[0059] Where γ0 represents the angle between the camera and the horizontal direction of the ground.

[0060] Specifically, the formula for calculating the intercept of the first line equation is as follows:

[0061] b1 = y i +tanγ0·x i .

[0062] Specifically, the formula for calculating the slope of the second line equation is as follows:

[0063]

[0064] Among them, (x c ,y c ) represents the third coordinate.

[0065] Specifically, the formula for calculating the intercept of the second line equation is as follows:

[0066]

[0067] Step S108: Calculate the corrected pitch angle of the camera based on the slope of the first line equation and the slope of the second line equation, and calculate the corrected mounting height of the camera based on the slope of the second line equation, the intercept of the second line equation, and the coordinates of the point where the camera is located.

[0068] by Figure 4 For example, by calculating the angle between line IJ and BC, the pitch angle of the camera on the road segment BC can be obtained.

[0069] Specifically, the formula for calculating the corrected pitch angle of the camera is as follows:

[0070]

[0071] Where γ′ represents the camera's corrected pitch angle.

[0072] Specifically, the formula for calculating the corrected installation height of the camera is as follows:

[0073]

[0074] Where h′ represents the corrected installation height of the camera.

[0075] By following the steps in the above embodiments, the extrinsic parameters of the camera can be obtained, and the actual length of road surface defects (such as cracks) can be obtained from these parameters. Specifically, this can be calculated through the following process:

[0076] The coordinates of point P on the road surface in the road surface coordinate system are (X... p ,Y p ), which is related to the coordinates p(x) of point P in the image. p ,y p The correspondence between ) is as follows:

[0077]

[0078] The coefficients k1 to k5 are calculated using the following formulas:

[0079]

[0080] Where W and H represent the horizontal and vertical pixel counts of the camera, respectively, and are internal parameters of the camera. Given that the horizontal field of view of the camera is 2β and the vertical field of view is 2α, then β represents the horizontal field of view of half the camera, and α represents the vertical field of view of half the camera.

[0081] Obtain the coordinates of two points in the image (e.g., the two endpoints of a crack), convert them to the coordinates of the two points in the road surface coordinate system using the above formula, and calculate the actual length of the two points (e.g., the length of the crack) using the distance calculation formula between the two points and their coordinates in the road surface coordinate system.

[0082] This invention also discloses a computer-readable storage medium storing computer program instructions; when executed by a processor, the computer program instructions implement the camera extrinsic parameter correction method for road surface detection as described in the above embodiments.

[0083] This invention also discloses a camera extrinsic parameter correction system for road surface detection, comprising: a computer-readable storage medium as described in the above embodiments.

[0084] In summary, the embodiments of the present invention correct the errors existing in the practical application of the fixed external parameter description camera photogrammetry system model by collecting the road surface longitudinal profile elevation, vehicle travel distance and equipment attitude information. It does not need to rely on road surface reference objects and has better applicability.

[0085] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for correcting camera extrinsic parameters for road surface detection, characterized in that, include: While the vehicle is traveling on the road, the camera's optical axis is used to collect data in real time, including the angle between the camera's optical axis and the horizontal direction of the ground, the elevation of the road's longitudinal profile, and the distance the vehicle travels. With the starting point of the road surface as the origin, the horizontal direction as the horizontal axis, the vertical direction as the vertical axis, the horizontal component of the vehicle's travel distance as the horizontal coordinate, and the elevation of the road surface longitudinal section as the vertical coordinate, a broken line is drawn to obtain a broken line graph. When the vehicle travels to a certain position, the first coordinate of the starting point and the second coordinate of the ending point of the straight line segment where the vehicle is located in the polyline graph are obtained, as well as the third coordinate of the ending point of the straight line segment where the intersection of the camera optical axis corresponding to the camera location and the polyline of the polyline graph is located. Based on the first coordinate, the second coordinate, and the vehicle's travel mileage, calculate the coordinates of the point where the vehicle is located in the line graph; The coordinates of the camera in the polyline graph are calculated based on the first coordinate, the second coordinate, the coordinates of the vehicle location, and the camera installation height. Establish the first straight line equation corresponding to the camera optical axis in the polyline graph, and the second straight line equation corresponding to the straight line segment where the intersection of the camera optical axis at the point where the camera is located and the polyline of the polyline graph is located. The slope of the first straight line equation is determined based on the angle between the camera and the horizontal direction of the ground, and the slope and intercept of the second straight line equation are determined based on the second coordinate and the third coordinate. The corrected pitch angle of the camera is calculated based on the slope of the first line equation and the slope of the second line equation, and the corrected mounting height of the camera is calculated based on the slope of the second line equation, the intercept of the second line equation, and the coordinates of the point where the camera is located.

2. The camera extrinsic parameter correction method for road surface detection according to claim 1, characterized in that, The equation of the first straight line is: y = a1x + b1; Where a1 represents the slope of the first straight line equation, b1 represents the intercept of the first straight line equation, a1 = -tanγ0, and γ0 represents the angle between the camera and the horizontal direction of the ground.

3. The camera extrinsic parameter correction method for road surface detection according to claim 2, characterized in that, The equation of the second straight line is: y = a²x + b²; Where a2 represents the slope of the second line equation, and b2 represents the intercept of the second line equation. (x b ,y b (x) represents the second coordinate, (x) c ,y c ) represents the third coordinate.

4. The camera extrinsic parameter correction method for road surface detection according to claim 3, characterized in that, The formula for calculating the corrected pitch angle of the camera is: Wherein, γ′ represents the corrected pitch angle of the camera.

5. The camera extrinsic parameter correction method for road surface detection according to claim 3, characterized in that, The formula for calculating the corrected installation height of the camera is: Where h′ represents the corrected installation height of the camera, (x i ,y i ) represents the coordinates of the point where the camera is located.

6. The camera extrinsic parameter correction method for road surface detection according to claim 5, characterized in that, The formula for calculating the coordinates of the point where the camera is located is: Among them, (x a ,y a (x) represents the first coordinate, (x) h ,y h ) represents the coordinates of the location of the vehicle, and h represents the installation height of the camera.

7. The camera extrinsic parameter correction method for road surface detection according to claim 6, characterized in that, The formula for calculating the coordinates of the location of the vehicle is: Wherein, s represents the absolute value of the difference between the vehicle's location and the starting point of the straight line segment on which the vehicle's location is situated, corresponding to the vehicle's travel distance.

8. The camera extrinsic parameter correction method for road surface detection according to claim 6, characterized in that, The formula for calculating the x-coordinate of the endpoint of each straight line segment of the line graph is as follows: Where (x1, y1) represents the coordinates of the starting point of a straight line segment in the line graph, (x2, y2) represents the coordinates of the ending point of a straight line segment in the line graph, and d represents the distance between the starting point and the ending point of the straight line segment. The ordinate of the endpoint of each straight line segment in the broken line graph is the elevation of its respective road surface longitudinal profile.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by a processor, they implement the camera extrinsic parameter correction method for road surface detection as described in any one of claims 1 to 8.

10. A camera extrinsic parameter correction system for road surface detection, characterized in that, include: The computer-readable storage medium as described in claim 9.

Images