Monocular camera three-dimensional lane line perception method, system and electronic device

Abstract

A method, system, and electronic device for 3D lane line perception using a monocular camera are disclosed. The method includes: acquiring 2D image information based on a monocular camera; randomly acquiring a first set of points located longitudinally in front of the vehicle in a ground coordinate system, and reprojecting the first set of points onto the camera coordinate system to form a second set of points; acquiring a third set of points located on the lane line in the camera coordinate system, corresponding to the second set of points and within the same time frame; calculating the difference in the lateral coordinates of corresponding points in the two sets of points, summing all differences to form a reprojection error, and including the reprojection error in an observation error function; optimizing the observation error function by minimizing the second set of points, and acquiring the first set of points corresponding to the second set of points when the observation error function is minimized, as 3D lane line points; and fitting a first lane line function based on the 3D lane line points. This method decouples monocular 3D lane line perception from camera extrinsic parameter calibration and ground plane position priors.

Description

Technical Field

[0001] This invention relates to the field of lane line detection technology, and in particular to a monocular camera-based three-dimensional lane line perception method, system, and electronic device. Background Technology

[0002] Lane detection and tracking are indispensable components of autonomous driving technology. In areas lacking high-precision maps or with poor localization, lane detection and tracking can be used by the planning and control (PNC) module to guide autonomous vehicles along lane lines, and can also be used for lateral localization by the localization module.

[0003] Monocular cameras have been widely used in the field of autonomous driving due to their advantages such as low cost and rich semantic information. Therefore, monocular 3D lane line perception has always been a core issue in the field of autonomous driving.

[0004] Since monocular cameras cannot directly acquire 3D information, obtaining 3D lane line information using monocular sensors has always been a challenging problem. Currently, most industry methods use camera calibration extrinsic parameters and ground plane position priors to calculate lane line positions, but this method has serious shortcomings. First, vehicles experience bumps during driving, causing changes in camera extrinsic parameters; second, ground plane position priors are invalid in areas such as uphill and downhill sections. Therefore, the 3D lane lines obtained using this method will have positional deviations, affecting driving safety.

[0005] The information disclosed in this background section is intended only to enhance the understanding of the general background of the invention and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0006] To address the problems existing in the prior art, this invention provides a method, system, and electronic device for three-dimensional lane line perception using a monocular camera.

[0007] This invention provides a method for three-dimensional lane line perception using a monocular camera, the method comprising:

[0008] Two-dimensional image information is acquired based on a monocular camera, and the two-dimensional image information includes lane lines;

[0009] Randomly obtain the first set of points located longitudinally in front of the vehicle in the ground coordinate system, and reproject the first set of points onto the camera coordinate system to form the second set of points;

[0010] Based on the two-dimensional image information, a third set of points located on the lane line in the camera coordinate system, corresponding to the second set of points and located in the same time frame, is obtained.

[0011] Based on the second set of points and the third set of points, calculate the difference in the lateral coordinates of corresponding points in the two sets of points, add all the differences to form the reprojection error, and include the reprojection error in the observation error function;

[0012] With the minimum observation error function as the optimization objective, the first set of points corresponding to the second set of points when the observation error function is minimized is obtained and used as three-dimensional lane line points;

[0013] Based on the three-dimensional lane line points, a first lane line function is fitted to form.

[0014] According to the present invention, a three-dimensional lane line perception method using a monocular camera is provided, which acquires two-dimensional image information based on a monocular camera, including:

[0015] Two-dimensional image information of multiple frames is acquired based on a monocular camera;

[0016] Correspondingly,

[0017] Randomly acquire a first set of points located longitudinally in front of the vehicle in the ground coordinate system, and reproject the first set of points onto the camera coordinate system to form a second set of points, including:

[0018] Randomly obtain the first set of points located longitudinally in front of the vehicle in the current frame's ground coordinate system;

[0019] Based on the first set of points in the current frame, coordinate transformation is performed to obtain multiple first set of points corresponding to the multiple frames;

[0020] The multiple first group points are reprojected onto the camera coordinate system of the corresponding frame to form multiple second group points;

[0021] Correspondingly,

[0022] Based on the second set of points and the third set of points, the difference in the lateral coordinates of corresponding points in the two sets of points is calculated. All differences are summed to form the reprojection error, which is then included in the observation error function, including:

[0023] For each time frame in the plurality of frames, based on the second set of points and the third set of points in the time frame, the difference in the horizontal coordinates of corresponding points in the two sets of points is calculated, and the sum of all the differences in the time frame is calculated.

[0024] The reprojection error is formed by summing the differences of all the time frames.

[0025] The reprojection error is included in the observation error function.

[0026] According to the present invention, a three-dimensional lane line perception method using a monocular camera is provided, which acquires two-dimensional image information based on a monocular camera. The two-dimensional image information includes lane lines, comprising:

[0027] Two-dimensional image information is acquired based on a monocular camera, and the two-dimensional image information includes multiple parallel lane lines;

[0028] Correspondingly,

[0029] Randomly obtain the first set of points located longitudinally in front of the vehicle in the ground coordinate system, including:

[0030] Randomly obtain a first set of points located longitudinally in front of the vehicle in the ground coordinate system. The first set of points includes multiple sub-sets of points corresponding to the multiple parallel lane lines.

[0031] Correspondingly, the method also includes:

[0032] Based on multiple subgroup points, multiple second lane line functions are fitted to form;

[0033] Calculate the average parameter of the non-constant terms of the actual parameters of the plurality of second lane line functions;

[0034] The degree of dispersion between the actual parameters of the non-constant terms of the plurality of second lane line functions and the average parameters is calculated as the parallel error;

[0035] The parallelism error is included in the observation error function.

[0036] According to the present invention, a three-dimensional lane line perception method using a monocular camera is provided, which acquires two-dimensional image information based on a monocular camera. The two-dimensional image information includes multiple parallel lane lines, including:

[0037] Determine whether the two-dimensional image information includes a branch road;

[0038] If the two-dimensional image information is determined to include a fork in the road, then based on the principle that parallel lane lines have the same vanishing point, multiple parallel lane lines are grouped together.

[0039] According to a monocular camera three-dimensional lane line perception method provided by the present invention, the method further includes:

[0040] Based on the first set of points, a third lane line function is fitted to form;

[0041] Calculate the fitting error between the first set of points and the third lane line function;

[0042] The fitting error is included in the observation error function.

[0043] According to the present invention, a monocular camera three-dimensional lane line perception method is provided, the method comprising:

[0044] The reprojection error, the fitting error, and the parallelism error are weighted and summed to form the observation error function.

[0045] A monocular camera-based three-dimensional lane line perception method according to the present invention, with the minimization of the observation error function as the optimization objective, includes:

[0046] The observation error function is optimized using the gradient descent method.

[0047] This invention also provides a monocular camera-based three-dimensional lane line perception system, the system comprising:

[0048] An image acquisition module is used to acquire two-dimensional image information based on a monocular camera, the two-dimensional image information including lane lines;

[0049] The reprojection point acquisition module is used to randomly acquire the first set of points located longitudinally in front of the vehicle in the ground coordinate system, and reproject the first set of points to the camera coordinate system to form the second set of points.

[0050] The original image point acquisition module is used to acquire, based on the two-dimensional image information, a third set of points located on the lane line in the camera coordinate system, corresponding to the second set of points and located in the same time frame;

[0051] The reprojection error acquisition module is used to calculate the difference in the lateral coordinates of corresponding points in the second set of points and the third set of points, add all the differences to form the reprojection error, and include the reprojection error in the observation error function.

[0052] The optimization module is used to obtain the first set of points corresponding to the second set of points when the observation error function is minimized, with the minimum observation error function as the optimization objective, and use them as three-dimensional lane line points;

[0053] The fitting module is used to fit a first lane line function based on the three-dimensional lane line points.

[0054] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the monocular camera three-dimensional lane line perception method as described in any of the preceding claims.

[0055] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the monocular camera three-dimensional lane line perception method as described in any of the preceding claims.

[0056] The monocular camera 3D lane line perception method, system, and electronic device provided by this invention, combined with the 3D lane line reprojection error, transforms the 3D lane line perception problem into a function optimization problem, and solves the problem by constructing a computational graph, thereby achieving decoupling of monocular 3D lane line perception from camera extrinsic parameter calibration and ground plane position prior. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0058] Figure 1 This invention provides a schematic flowchart of a monocular camera-based three-dimensional lane line perception method.

[0059] Figure 2 This invention also provides a schematic diagram of a monocular camera three-dimensional lane line perception system.

[0060] Figure 3 A schematic diagram comparing the application effects of prior art and the method of the present invention;

[0061] Figure 4 This is a schematic diagram of the physical structure of an electronic device provided by the present invention. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0063] The monocular camera 3D lane line perception method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0064] It should be noted that this invention relates to three coordinate systems: the vehicle coordinate system, the camera coordinate system, and the ground coordinate system.

[0065] Specifically, a vehicle coordinate system is established with the vertical projection point of the rear axle center onto the ground as the origin, where the x-axis points directly in front of the vehicle, the y-axis points to the left side of the vehicle, and the z-axis points to the sky.

[0066] Specifically, a ground coordinate system is established with the origin of the vehicle coordinate system in the current frame as the origin, and the projection of the vehicle coordinate system's x-axis onto the ground plane as the x-axis. Its y-axis is perpendicular to the x-axis to the right on the ground plane, and its z-axis is perpendicular to the ground plane and points upwards. Ideally, the vehicle coordinate system and the ground coordinate system are completely coincident. However, when the vehicle experiences bumps or is on inclines or declines, there are three Euler angle deviations between the vehicle coordinate system and the ground coordinate system. The errors in yaw, pitch, and roll angles are denoted as Δy, Δp, and Δr, respectively.

[0067] Furthermore, the lane line function fitted by this invention is a spiral curve or a straight line, and within a certain range, it can be fitted using a cubic curve. Let the total number of observed lane lines be n. In the current frame's ground coordinate system, the equation of the i-th lane line is denoted as follows:

[0068] y i =w i0 +w i1 *x+w i2 *x 2 +w i3 *x 3

[0069] Multiple frames are observed for n lane lines, with an observation window length of k (i.e., k time frames). Lane lines are static objects, and their functions are time-invariant. However, the Euler angle deviation between the vehicle coordinate system and the ground coordinate system varies from frame to frame. The cubic equation parameters of the n lane lines in the current frame's ground coordinate system, along with the Euler angle deviation between the vehicle coordinate system and the ground coordinate system in each frame, are denoted as the parameters to be solved, as shown in the following equation:

[0070] θ

[0071] =(w 10 ,w 11 ,w 12 ,w 13 …w n0 ,w n1 ,w n2 ,w n3 ,Δy0,Δp0,Δr0…Δy k ,Δp k ,Δr k )

[0072] Figure 1 This is a schematic diagram of a monocular camera-based three-dimensional lane line perception method provided by the present invention, as shown below. Figure 1 As shown, the present invention provides a monocular camera three-dimensional lane line perception method, which may include the following steps.

[0073] Preferably, the monocular camera three-dimensional lane line perception method of the present invention is applied to autonomous vehicles.

[0074] Preferably, in the following embodiment, the first set of points are 3D lane line points from the top-view perspective, the second set of points includes the reprojection points of the 3D lane line points from the top-view perspective on the image, and the third set of points includes 2D lane line pixels obtained using a 2D lane line detection network.

[0075] S100: Acquire two-dimensional image information based on a monocular camera. The two-dimensional image information includes lane lines.

[0076] S200. Randomly acquire the first set of points located longitudinally in front of the vehicle in the ground coordinate system, and reproject the first set of points onto the camera coordinate system to form the second set of points.

[0077] S300: Based on two-dimensional image information, obtain a third set of points located on the lane line in the camera coordinate system, corresponding to the second set of points and located in the same time frame.

[0078] S400. Based on the second and third sets of points, calculate the difference in the lateral coordinates of corresponding points in the two sets of points, add all the differences to form the reprojection error, and include the reprojection error in the observation error function.

[0079] S500. With the minimum observation error function as the optimization objective, obtain the first set of points corresponding to the second set of points when the observation error function is minimized, and use them as three-dimensional lane line points.

[0080] S600, based on three-dimensional lane line points, fits to form the first lane line function.

[0081] Optionally, two-dimensional image information is acquired based on a monocular camera, including:

[0082] Two-dimensional image information of multiple frames is acquired based on a monocular camera;

[0083] Correspondingly,

[0084] Randomly acquire the first set of points located longitudinally in front of the vehicle in the ground coordinate system. Reproject the first set of points into the camera coordinate system to form the second set of points, including:

[0085] Randomly obtain the first set of points located longitudinally in front of the vehicle in the current frame's ground coordinate system;

[0086] Based on the first set of points in the current frame, coordinate transformation is used to generate multiple first set of points corresponding to multiple frames;

[0087] Multiple first-group points are reprojected onto the camera coordinate system of the corresponding frame to form multiple second-group points;

[0088] Correspondingly,

[0089] Based on the second and third sets of points, calculate the difference in the lateral coordinates of corresponding points in the two sets, sum all differences to form the reprojection error, and include the reprojection error in the observation error function, including:

[0090] For each time frame in multiple frames, based on the second and third sets of points in the time frame, calculate the difference in the horizontal coordinates of corresponding points in the two sets of points, and calculate the sum of all differences in the time frame;

[0091] The reprojection error is formed by summing the differences of all time frames.

[0092] The reprojection error is included in the observation error function.

[0093] Preferably, a lane line detection neural network is used to detect two-dimensional lane lines from the input two-dimensional image.

[0094] Preferably, in the ground coordinate system, a set of three-dimensional lane line points are randomly generated at certain intervals in the x-direction (the initially randomly generated points may be completely different from or very similar to the true values). The three-dimensional lane line points are then reprojected onto the two-dimensional image using camera calibration parameters. When the positions of the three-dimensional lane line points differ from the true values, or when there is an Euler angle deviation between the vehicle coordinate system and the ground coordinate system, a reprojection error exists between the projection of the three-dimensional lane lines and the lane line detection results in the two-dimensional image. The formula for the reprojection error is as follows:

[0095]

[0096] X i =R i2k X+T i2k

[0097] ΔR i =f(Δy) i ,Δp i ,Δr i )

[0098] Where K is the camera intrinsic parameter; R v2c and T v2c This refers to the rotation matrix and translation vector between the vehicle coordinate system and the camera coordinate system. The rotation matrix and translation vector are the camera extrinsic parameters. This invention uses the default extrinsic parameters when the vehicle is stationary, and continues to use these parameters even during vehicle movement. By optimizing the function, the error caused by changes in these extrinsic parameters is reduced, thus achieving decoupling from the camera extrinsic parameters; ΔR i U is the rotation matrix between the vehicle coordinate system and the ground coordinate system in the i-th frame, which is calculated from the Euler angle deviation between the vehicle coordinate system and the ground coordinate system in the i-th frame; iX is the pixel coordinate of the lane line point detected by the lane line neural network in the i-th frame (in the camera coordinate system); X is the 3D lane line point coordinate in the ground coordinate system of the current frame (the k-th frame). i R is the coordinate of the three-dimensional lane line points in the ground coordinate system of the i-th frame, calculated using positioning information. i2k and T i2k Based on the rotation matrix and translation vector from the i-th frame to the current frame (k-th frame) obtained from the positioning information, s is a scaling factor equal to the reciprocal of the lane line point depth (i.e., the absolute value of the Z coordinate of the lane line point in the camera coordinate system).

[0099] The above formula essentially transforms the coordinates of the 3D lane line points in the ground coordinate system of the current frame to the camera coordinate system of the previous 1-k frames, and sums the errors with the detected lane line pixel coordinates in the camera coordinate system of the previous 1-k frames, which is then used as the reprojection error.

[0100] Optionally, two-dimensional image information is acquired based on a monocular camera. The two-dimensional image information includes lane lines, including:

[0101] Two-dimensional image information is acquired using a monocular camera, and the two-dimensional image information includes multiple parallel lane lines;

[0102] Correspondingly,

[0103] Randomly obtain the first set of points located longitudinally in front of the vehicle in the ground coordinate system, including:

[0104] Randomly obtain the first set of points located longitudinally in front of the vehicle in the ground coordinate system. The first set of points includes multiple sub-sets of points corresponding to multiple parallel lane lines.

[0105] Correspondingly, the methods also include:

[0106] Based on multiple subgroup points, multiple second lane line functions are fitted to form;

[0107] Calculate the average parameter of the non-constant terms of the actual parameters of multiple second lane line functions;

[0108] The degree of dispersion between the actual parameters and the average parameters of the non-constant terms of multiple second lane line functions is calculated as the parallel error;

[0109] Parallelism error is included in the observation error function.

[0110] It should be noted that for n parallel lane lines, it can be approximated that the non-constant terms in their parametric equations should remain consistent. After calculating the average parameter terms of the parallel lane line parametric equations, the parallelism error can be determined using the following formula:

[0111]

[0112] Wi =(w i0 w i1 w i2 w i3 )

[0113] Optionally, two-dimensional image information is acquired based on a monocular camera. The two-dimensional image information includes multiple parallel lane lines, including:

[0114] Determine whether the two-dimensional image information includes branch roads;

[0115] If the two-dimensional image information is determined to include forks, then multiple parallel lane lines are grouped based on the principle that parallel lane lines have the same vanishing point.

[0116] It should be noted that in most cases, lane lines are parallel. However, if a fork in the road is encountered, the lane lines within each group remain parallel after grouping. The lane lines are grouped based on the vanishing point principle, and parallel lane lines are obtained from the detected two-dimensional lane lines. Optionally, the method also includes:

[0117] Based on the first set of points, a third lane line function is fitted;

[0118] Calculate the fitting error of the first set of points and the third lane line function;

[0119] The fitting error is included in the observation error function.

[0120] It should be noted that the randomly generated lane line points should be distributed along the same cubic curve. Assuming each lane line generates m points, the fitting error between the n lane line points and the parametric equation can be calculated. Where (x... ij y ij ) represents the coordinates of the j-th lane line point on the i-th lane line. The specific fitting error is explained in the following formula:

[0121]

[0122] Optionally, the method includes:

[0123] The reprojection error, fitting error, and parallelism error are weighted and summed to form the observation error function.

[0124] It should be noted that the reprojection error, fitting error, and parallelism error are weighted and summed to define the observation error of the 3D lane line. This invention considers all three errors simultaneously, which significantly improves the lane line perception accuracy. The method involves solving for the parameters that minimize the observation error. Finally, the ideal three-dimensional lane line parameter equation can be obtained, and the optimization formula is as follows:

[0125] error = error fit +error parallel+error reproj

[0126]

[0127] Optionally, the optimization objective is to minimize the observation error function, including:

[0128] The observation error function is optimized using the gradient descent method.

[0129] It should be noted that this invention transforms the problem of solving three-dimensional lane lines into a nonlinear optimization problem, and uses the gradient descent method to solve this optimization problem. After the calculation converges, the optimal lane line function can be obtained. Furthermore, since this lane line function is based on the ground coordinate system, the ground position of the lane line can also be accurately obtained.

[0130] This embodiment utilizes the parallel characteristics of lane lines, combined with the reprojection error of 3D lane lines and the fitting error between the fitting point and the fitting function, to transform the 3D lane line perception problem into a function optimization problem. The problem is solved by constructing a computational graph, thus achieving the decoupling of monocular 3D lane line perception from camera extrinsic parameter calibration and ground plane position prior.

[0131] The monocular camera three-dimensional lane line perception system provided by the present invention is described below. The monocular camera three-dimensional lane line perception system described below can be referred to in correspondence with the monocular camera three-dimensional lane line perception method described above.

[0132] Figure 2 This invention also provides a schematic diagram of a monocular camera-based three-dimensional lane line perception system, as shown below. Figure 2 As shown, the present invention also provides a monocular camera three-dimensional lane line perception system, the system comprising:

[0133] The image acquisition module is used to acquire two-dimensional image information based on a monocular camera. The two-dimensional image information includes lane lines.

[0134] The reprojection point acquisition module is used to randomly acquire the first set of points located longitudinally in front of the vehicle in the ground coordinate system, and reproject the first set of points onto the camera coordinate system to form the second set of points.

[0135] The original image point acquisition module is used to acquire, based on two-dimensional image information, a third set of points located on the lane line in the camera coordinate system, corresponding to the second set of points and located in the same time frame;

[0136] The reprojection error acquisition module is used to calculate the difference in the lateral coordinates of corresponding points in the second and third sets of points, sum all the differences to form the reprojection error, and include the reprojection error in the observation error function.

[0137] The optimization module is used to obtain the first set of points corresponding to the second set of points when the observation error function is minimized, with the goal of minimizing the observation error function, and use them as the three-dimensional lane line points.

[0138] The fitting module is used to fit a first lane line function based on the three-dimensional lane line points.

[0139] This embodiment combines the 3D lane line reprojection error to transform the 3D lane line perception problem into a function optimization problem. The problem is solved by constructing a computational graph, thus achieving the decoupling of monocular 3D lane line perception from camera extrinsic parameter calibration and ground plane position prior.

[0140] The present invention also provides a vehicle including the monocular camera three-dimensional lane line perception system.

[0141] It should be noted that the proposed perception method and the traditional method were executed on datasets containing a large number of inclines, declines, and bumps to obtain lane line functions. The lateral errors between the lane line position and the true position at different longitudinal positions were statistically analyzed and used as a basis for evaluating lane line performance. Figure 3 A comparative diagram of the application effects of the prior art and the method of the present invention is shown below. Figure 3 As shown, the results indicate that the present invention can effectively reduce lane line position errors and has excellent usability.

[0142] Figure 4 A schematic diagram of the physical structure of an electronic device provided by the present invention, such as... Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a monocular camera three-dimensional lane line perception method, the method including:

[0143] Two-dimensional image information is acquired based on a monocular camera, and the two-dimensional image information includes lane lines;

[0144] Randomly obtain the first set of points located longitudinally in front of the vehicle in the ground coordinate system, and reproject the first set of points onto the camera coordinate system to form the second set of points;

[0145] Based on the two-dimensional image information, a third set of points located on the lane line in the camera coordinate system, corresponding to the second set of points and located in the same time frame, is obtained.

[0146] Based on the second set of points and the third set of points, calculate the difference in the lateral coordinates of corresponding points in the two sets of points, add all the differences to form the reprojection error, and include the reprojection error in the observation error function;

[0147] With the minimum observation error function as the optimization objective, the first set of points corresponding to the second set of points when the observation error function is minimized is obtained and used as three-dimensional lane line points;

[0148] Based on the three-dimensional lane line points, a first lane line function is fitted to form.

[0149] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0150] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, wherein when the program instructions are executed by a computer, the computer is able to execute the monocular camera three-dimensional lane line perception method provided by the above methods, the method comprising:

[0151] Two-dimensional image information is acquired based on a monocular camera, and the two-dimensional image information includes lane lines;

[0152] Randomly obtain the first set of points located longitudinally in front of the vehicle in the ground coordinate system, and reproject the first set of points onto the camera coordinate system to form the second set of points;

[0153] Based on the two-dimensional image information, a third set of points located on the lane line in the camera coordinate system, corresponding to the second set of points and located in the same time frame, is obtained.

[0154] Based on the second set of points and the third set of points, calculate the difference in the lateral coordinates of corresponding points in the two sets of points, add all the differences to form the reprojection error, and include the reprojection error in the observation error function;

[0155] With the minimum observation error function as the optimization objective, the first set of points corresponding to the second set of points when the observation error function is minimized is obtained and used as three-dimensional lane line points;

[0156] Based on the three-dimensional lane line points, a first lane line function is fitted to form.

[0157] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the monocular camera three-dimensional lane line perception methods provided above, the methods comprising:

[0158] Two-dimensional image information is acquired based on a monocular camera, and the two-dimensional image information includes lane lines;

[0159] Randomly obtain the first set of points located longitudinally in front of the vehicle in the ground coordinate system, and reproject the first set of points onto the camera coordinate system to form the second set of points;

[0160] Based on the two-dimensional image information, a third set of points located on the lane line in the camera coordinate system, corresponding to the second set of points and located in the same time frame, is obtained.

[0161] Based on the second set of points and the third set of points, calculate the difference in the lateral coordinates of corresponding points in the two sets of points, add all the differences to form the reprojection error, and include the reprojection error in the observation error function;

[0162] With the minimum observation error function as the optimization objective, the first set of points corresponding to the second set of points when the observation error function is minimized is obtained and used as three-dimensional lane line points;

[0163] Based on the three-dimensional lane line points, a first lane line function is fitted to form.

[0164] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0165] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0166] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A monocular camera three-dimensional lane line perception method, characterized in that, The method includes: Two-dimensional image information is acquired based on a monocular camera, and the two-dimensional image information includes lane lines; In the ground coordinate system, along the X-axis direction, the first set of points located longitudinally in front of the vehicle in the ground coordinate system are randomly obtained at certain intervals. The first set of points is then reprojected onto the camera coordinate system to form the second set of points. Based on the two-dimensional image information, a third set of points located on the lane line in the camera coordinate system, corresponding to the second set of points and located in the same time frame, is obtained. Based on the second set of points and the third set of points, calculate the difference in the lateral coordinates of corresponding points in the two sets of points in the camera coordinate system, add all the differences to form the reprojection error, and include the reprojection error in the observation error function. With the minimum observation error function as the optimization objective, the first set of points corresponding to the second set of points when the observation error function is minimized is obtained and used as three-dimensional lane line points; Based on the three-dimensional lane line points, a first lane line function is fitted to form.

2. The monocular camera three-dimensional lane line perception method according to claim 1, characterized in that, Two-dimensional image information acquired using a monocular camera includes: Two-dimensional image information of multiple frames is acquired based on a monocular camera; Correspondingly, In the ground coordinate system, a first set of points located longitudinally in front of the vehicle along the X-axis at certain intervals is randomly acquired. These first set of points are then reprojected onto the camera coordinate system to form a second set of points, including: In the ground coordinate system, along the X-axis direction, the first set of points located longitudinally in front of the vehicle in the current frame's ground coordinate system is randomly acquired at certain intervals. Based on the first set of points in the current frame, coordinate transformation is performed to obtain multiple first set of points corresponding to the multiple frames; The multiple first group points are reprojected onto the camera coordinate system of the corresponding frame to form multiple second group points; Correspondingly, Based on the second set of points and the third set of points, calculate the difference in the lateral coordinates of corresponding points in the two sets of points in the camera coordinate system, sum all differences to form the reprojection error, and include the reprojection error in the observation error function, including: For each time frame in the plurality of frames, based on the second set of points and the third set of points in the time frame, the difference in the horizontal coordinates of corresponding points in the two sets of points is calculated, and the sum of all the differences in the time frame is calculated. The reprojection error is formed by summing the differences of all the time frames. The reprojection error is included in the observation error function.

3. The monocular camera three-dimensional lane line perception method according to claim 1 or 2, characterized in that, Two-dimensional image information is acquired based on a monocular camera, and the two-dimensional image information includes lane lines, including: Two-dimensional image information is acquired based on a monocular camera, and the two-dimensional image information includes multiple parallel lane lines; Correspondingly, In the ground coordinate system, the first set of points located longitudinally in front of the vehicle along the X-axis direction at certain intervals are randomly selected, including: In the ground coordinate system, along the X-axis direction, at certain intervals, a first set of points located longitudinally in front of the vehicle in the ground coordinate system is randomly obtained. The first set of points includes multiple sub-sets of points corresponding to the multiple parallel lane lines. Correspondingly, the method also includes: Based on multiple subgroup points, multiple second lane line functions are fitted to form; Calculate the average parameter of the non-constant terms of the actual parameters of the plurality of second lane line functions; The degree of dispersion between the actual parameters and the average parameters of the non-constant terms of the plurality of second lane line functions is calculated as the parallel error; The parallelism error is included in the observation error function.

4. The monocular camera three-dimensional lane line perception method according to claim 3, characterized in that, Two-dimensional image information is acquired based on a monocular camera, and the two-dimensional image information includes multiple parallel lane lines, including: Determine whether the two-dimensional image information includes a branch road; If the two-dimensional image information is determined to include a fork in the road, then based on the principle that parallel lane lines have the same vanishing point, multiple parallel lane lines are grouped together.

5. The monocular camera three-dimensional lane line perception method according to claim 3, characterized in that, The method further includes: Based on the first set of points, a third lane line function is fitted to form; Calculate the fitting error between the first set of points and the third lane line function; The fitting error is included in the observation error function.

6. The monocular camera three-dimensional lane line perception method according to claim 5, characterized in that, The method includes: The reprojection error, the fitting error, and the parallelism error are weighted and summed to form the observation error function.

7. The monocular camera three-dimensional lane line perception method according to any one of claims 1-6, characterized in that, The optimization objective is to minimize the observation error function, including: The observation error function is optimized using the gradient descent method.

8. A monocular camera-based three-dimensional lane line perception system, characterized in that, The system includes: An image acquisition module is used to acquire two-dimensional image information based on a monocular camera, the two-dimensional image information including lane lines; The reprojection point acquisition module is used to randomly acquire a first set of points located longitudinally in front of the vehicle in the ground coordinate system along the X-axis direction at certain intervals, and reproject the first set of points to the camera coordinate system to form a second set of points. The original image point acquisition module is used to acquire, based on the two-dimensional image information, a third set of points located on the lane line in the camera coordinate system, corresponding to the second set of points and located in the same time frame; The reprojection error acquisition module is used to calculate the difference in the lateral coordinates of corresponding points in the two sets of points in the camera coordinate system based on the second set of points and the third set of points, add all the differences to form the reprojection error, and include the reprojection error in the observation error function. The optimization module is used to obtain the first set of points corresponding to the second set of points when the observation error function is minimized, with the minimum observation error function as the optimization objective, and use them as three-dimensional lane line points; The fitting module is used to fit a first lane line function based on the three-dimensional lane line points.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the monocular camera three-dimensional lane line perception method as described in any one of claims 1-7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the monocular camera three-dimensional lane line perception method as described in any one of claims 1-7.

Images