Lane line detection method and system based on obstacle detection and electronic equipment

Abstract

The embodiment of the invention provides a lane line detection method and system based on obstacle detection and electronic equipment, and the method can obtain a source image collected by an image collector in real time, and carries out the three-dimensional obstacle detection according to the source image, so as to output an obstacle detection result. And analyzing a grounding point of an obstacle in the obstacle detection result, constructing a road surface equation of a fitting road surface according to the grounding point, and obtaining a two-dimensional lane line detection result of the source image. And calculating a simulated three-dimensional coordinate of the lane line based on the pavement equation and the two-dimensional lane line detection result, constructing a lane line equation according to the simulated three-dimensional coordinate, and generating a three-dimensional lane line detection result through the lane line equation. According to the method, through fusion of three-dimensional obstacle detection and two-dimensional lane line detection, the accuracy of obstacle detection can be improved on the premise of reducing the complexity of lane line detection, so that the effect of lane line detection is improved.

Description

Technical Field

[0001] The present application relates to the field of three-dimensional detection technology, and in particular to a lane line detection method, system and electronic equipment based on obstacle detection. Background Art

[0002] Lane line perception technology is one of the key technologies in intelligent driving systems. It aims to obtain road image information in real time through sensors such as cameras and radars, and accurately identify and locate lane lines. Lane line perception technology identifies lane lines by collecting and analyzing features in road images, such as color, shape, texture, etc., and determines whether the vehicle deviates from the lane based on the identified lane lines. Lane line perception technology can be applied to fields such as autonomous driving, intelligent traffic management, and driver assistance systems, and can significantly improve the safety and efficiency of road traffic.

[0003] Lane line perception technology can detect three-dimensional lane lines based on deep learning methods, namely 3D lane line detection. For example, the three-dimensional information of lane lines is extracted from the input image data through a deep learning network (such as convolutional neural network CNN, Transformer, etc.). The network is trained with a large number of annotated 3D data sets to learn how to identify lane lines in images and predict their three-dimensional spatial positions. It has the ability to automatically extract features, high robustness and adapt to complex environments, so as to accurately output the three-dimensional coordinates of lane lines.

[0004] However, the annotation of the 3D data set used for the above lane line detection is relatively complex. For example, it is necessary to determine the precise position of the lane line in three-dimensional space, including parameters such as height, width, and direction. This increases the overall cost and complexity of lane line detection and requires a large amount of computing resources, thus affecting the applicability of lane line detection. Summary of the invention

[0005] The present application provides a lane line detection method, system and electronic device based on obstacle detection to solve the problem of high complexity of lane line detection.

[0006] In a first aspect, some embodiments of the present application provide a lane line detection method based on obstacle detection, comprising the following steps:

[0007] Acquire a source image, where the source image is a real-time image acquired by an image collector;

[0008] Performing three-dimensional obstacle detection according to the source image to output an obstacle detection result, wherein the obstacle detection result includes the three-dimensional coordinates of the obstacle;

[0009] parsing a grounding point of the obstacle in the obstacle detection result;

[0010] Constructing a pavement equation for fitting the pavement according to the grounding point;

[0011] Acquire a two-dimensional lane line detection result, where the two-dimensional lane line detection result is a detection result of performing two-dimensional lane line detection on the source image, and the two-dimensional lane line detection result includes pixel coordinates of the lane line;

[0012] Calculating the simulated three-dimensional coordinates of the lane line based on the road surface equation and the two-dimensional lane line detection result;

[0013] Constructing a lane line equation according to the simulated three-dimensional coordinates;

[0014] A three-dimensional lane line detection result is generated by the lane line equation, and the three-dimensional lane line detection result includes the three-dimensional coordinates of the lane line.

[0015] In some embodiments of the present application, constructing a pavement equation for fitting the pavement according to the touchdown point includes: reading a first target sub-coordinate of the touchdown point in a first direction; calculating a first power form of the first target sub-coordinate according to a preset index, wherein the preset index includes 0, 1, 2, and 3; generating a first coefficient corresponding to the first power form; calculating a first product of the first power form and the corresponding first coefficient; calculating a first sum of the first products, and generating the pavement equation according to the first sum.

[0016] In some embodiments of the present application, after the step of constructing a pavement equation for fitting the road surface according to the grounding point, it also includes: generating a first reference parameter according to the pavement equation, the first reference parameter is generated based on the first coefficient; constructing a first objective function of the pavement equation, the goal of the first objective function is to minimize the sum of squares of the distance from the grounding point to the fitting road surface, the first objective function includes the fitting weight of the grounding point and the first reference parameter; solving the first objective function based on an optimization algorithm to obtain the value of the first reference parameter.

[0017] In some embodiments of the present application, the first objective function is shown as follows:

[0018]

[0019] In the formula, J r is the first objective function, ω i is the fitting weight of the grounding point, N is the number of grounding points, C r is the first reference parameter, λ r is a preset constant, z oi is the sub-coordinate of the ground point in a second direction, where the second direction is perpendicular to the first direction, equal x o is the sub-coordinate of the ground point in the first direction.

[0020] In some embodiments of the present application, the calculation of the simulated three-dimensional coordinates of the lane line based on the road surface equation and the two-dimensional lane line detection result includes: reading the internal and external parameters of the camera associated with the image collector; converting the pixel point coordinates of the lane line into reference three-dimensional coordinates according to the internal and external parameters of the camera and the two-dimensional lane line detection result; the pixel point coordinates include sub-coordinates in a first direction and a third direction, the first direction is perpendicular to the third direction, and the reference three-dimensional coordinates include unknown sub-coordinates of the pixel point coordinates and a second direction, the second direction is perpendicular to the first direction and the third direction; solving the unknown sub-coordinates according to the road surface equation to obtain the simulated three-dimensional coordinates of the lane line.

[0021] In some embodiments of the present application, after the step of reading the internal and external parameters of the camera associated with the image acquisition device, it also includes: parsing the corresponding intrinsic parameter matrix and extrinsic parameter matrix according to the internal and external parameters of the camera; constructing the transformation equation of the reference three-dimensional coordinates through the intrinsic parameter matrix, the extrinsic parameter matrix and the pixel point coordinates; and generating the reference three-dimensional coordinates through the transformation equation.

[0022] In some embodiments of the present application, constructing a lane line equation based on the simulated three-dimensional coordinates includes: reading a second target sub-coordinate of the reference three-dimensional coordinate in the first direction; calculating a second power form of the second target sub-coordinate according to a preset exponent, wherein the preset exponent includes 0, 1, 2, and 3; generating a second coefficient corresponding to the second power form; calculating a second product of the second power form and the corresponding second coefficient; calculating a second sum of the second product, and generating the lane line equation according to the second sum.

[0023] In some embodiments of the present application, after the step of constructing a lane line equation according to the simulated three-dimensional coordinates, it also includes: generating a second reference parameter according to the lane line equation, the second reference parameter is generated based on the second coefficient; constructing a second objective function of the lane line equation, the goal of the second objective function is to minimize the sum of squares of the distances from the simulated three-dimensional coordinates to the lane line equation, the second objective function includes the fitting weights of the simulated three-dimensional coordinates and the second reference parameter; solving the second objective function based on an optimization algorithm to obtain the value of the second reference parameter.

[0024] In a second aspect, some embodiments of the present application further provide a lane line detection system based on obstacle detection, including an input module, an obstacle perception module, a calculation module and a lane line perception module, wherein:

[0025] An input module is configured to acquire a source image, wherein the source image is a real-time image acquired by an image collector;

[0026] an obstacle sensing module, configured to perform three-dimensional obstacle detection according to the source image to output an obstacle detection result, wherein the obstacle detection result includes the three-dimensional coordinates of the obstacle;

[0027] A calculation module is configured to analyze the grounding point of the obstacle in the obstacle detection result; and construct a road surface equation for fitting the road surface according to the grounding point;

[0028] The lane line perception module is configured to obtain a two-dimensional lane line detection result, which is a detection result of performing two-dimensional lane line detection on the source image, and the two-dimensional lane line detection result includes the pixel coordinates of the lane line; calculate the simulated three-dimensional coordinates of the lane line based on the road surface equation and the two-dimensional lane line detection result; construct a lane line equation according to the simulated three-dimensional coordinates; and generate a three-dimensional lane line detection result through the lane line equation, and the three-dimensional lane line detection result includes the three-dimensional coordinates of the lane line.

[0029] In a third aspect, some embodiments of the present application further provide an electronic device, comprising a processor, a memory and a bus. The processor and the memory are communicatively connected via the bus; the memory stores computer program instructions executable by the processor, and the processor is configured to execute the following program steps:

[0030] Acquire a source image, where the source image is a real-time image acquired by an image collector;

[0031] Performing three-dimensional obstacle detection according to the source image to output an obstacle detection result, wherein the obstacle detection result includes the three-dimensional coordinates of the obstacle;

[0032] parsing a grounding point of the obstacle in the obstacle detection result;

[0033] Constructing a pavement equation for fitting the pavement according to the grounding point;

[0034] Acquire a two-dimensional lane line detection result, where the two-dimensional lane line detection result is a detection result of performing two-dimensional lane line detection on the source image, and the two-dimensional lane line detection result includes pixel coordinates of the lane line;

[0035] Calculating the simulated three-dimensional coordinates of the lane line based on the road surface equation and the two-dimensional lane line detection result;

[0036] Constructing a lane line equation according to the simulated three-dimensional coordinates;

[0037] A three-dimensional lane line detection result is generated by the lane line equation, and the three-dimensional lane line detection result includes the three-dimensional coordinates of the lane line.

[0038] It can be seen from the above technical solutions that some embodiments of the present application provide a lane line detection method, system and electronic device based on obstacle detection. The method can obtain the source image collected by the image collector in real time, and perform three-dimensional obstacle detection based on the source image to output the obstacle detection result. Then analyze the grounding point of the obstacle in the obstacle detection result, and construct a road surface equation for fitting the road surface according to the grounding point, and obtain the two-dimensional lane line detection result of the source image. Then calculate the simulated three-dimensional coordinates of the lane line based on the road surface equation and the result of the two-dimensional lane line detection, and construct a lane line equation based on the simulated three-dimensional coordinates, and generate a three-dimensional lane line detection result through the lane line equation. By integrating three-dimensional obstacle detection and two-dimensional lane line detection, the method can improve the accuracy of obstacle detection while reducing the complexity of lane line detection, thereby improving the effect of lane line detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 A schematic diagram of a process flow of a lane line detection method based on obstacle detection provided in some embodiments of the present application;

[0041] Figure 2 A schematic diagram of a process for constructing a road surface equation provided in some embodiments of the present application;

[0042] Figure 3 A schematic diagram of a process for calculating simulated three-dimensional coordinates provided in some embodiments of the present application;

[0043] Figure 4 A schematic diagram of a process for converting three-dimensional coordinates provided in some embodiments of the present application;

[0044] Figure 5 A schematic diagram of a solution process of a second objective function provided in some embodiments of the present application;

[0045] Figure 6 An architectural diagram of a lane detection system based on obstacle detection provided in some embodiments of the present application;

[0046] Figure 7 A schematic diagram of the structure of an electronic device provided for some embodiments of the present application. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of the exemplary embodiments of the present application clearer, the technical solutions in the exemplary embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0048] Based on the exemplary embodiments shown in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application. In addition, although the disclosure in this application is introduced according to one or several exemplary examples, it should be understood that each aspect of these disclosures can also constitute a complete technical solution separately.

[0049] It should be understood that the terms "first", "second", "third", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, for example, they can be implemented according to an order other than those given in the diagrams or descriptions of the embodiments of this application.

[0050] In addition, the terms "include" and "have" and any variations thereof are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to those components expressly listed but may include other components not expressly listed or inherent to such products or devices.

[0051] Lane line perception technology is to identify lane lines by collecting and analyzing features in road images, such as color, shape, texture, etc., and judge whether the vehicle deviates from the lane through the identified lane lines. For example, in an autonomous driving system, lane line perception is the basis for vehicle path planning and autonomous navigation. Through accurate lane line detection, autonomous driving vehicles can understand their position on the road in real time and make corresponding driving decisions, such as keeping lanes, changing lanes and overtaking. In addition, lane line perception technology can also provide real-time road condition information for intelligent traffic signal control systems, optimize signal light control strategies, and alleviate traffic congestion. In driving assistance systems, this technology can monitor and remind drivers to pay attention to lane deviation in real time to prevent traffic accidents.

[0052] In some embodiments, 3D lane line detection is performed based on a deep learning method. In particular, 3D lane line detection based on deep learning can use a deep learning network to directly extract the three-dimensional information of the lane line from the input image data, such as the three-dimensional information obtained based on the three-dimensional coordinates of the lane line. The deep learning network is trained with a large number of annotated 3D data sets to learn how to identify the lane lines in the image and predict their three-dimensional spatial positions.

[0053] However, the above-mentioned deep learning-based 3D lane line detection requires a large amount of 3D annotation data. The annotation process of 3D annotation data is relatively complicated. For example, it is necessary to determine the precise position of the lane line in three-dimensional space, including parameters such as height, width, and direction. This increases the overall cost and complexity of lane line detection and requires a large amount of computing resources, thus affecting the applicability of lane line detection.

[0054] Therefore, in some embodiments, 2D lane line detection is first performed using image data, and then a 3D lane line detection result with a road plane height value of zero is calculated using the result of the 2D lane line detection. For example, a road surface image is first captured using a camera, and lane lines are detected on a pixel plane using image processing techniques (such as edge detection, Hough transform, etc.). Subsequently, based on the road plane assumption (i.e., the lane lines are assumed to be on the same horizontal plane), the detected 2D lane lines are converted into 3D lane lines with a height value of zero.

[0055] However, the accuracy of converting 2D lane lines into 3D lane lines with a height value of zero is limited by the road plane assumption. In complex road environments such as ramps, bridges, and underbridges, the error of lane line detection will increase, affecting the accuracy of lane line detection.

[0056] To this end, some embodiments of the present application provide a lane line detection method based on obstacle detection, which can integrate 3D obstacle detection and 2D lane line detection, so that accurate 3D lane line detection results can be calculated based on the 3D obstacle detection results, so as to improve the accuracy of lane line detection while reducing the complexity of lane line detection.

[0057] like Figure 1 As shown, in some embodiments, the lane line detection method includes the following steps:

[0058] S1: Acquire a source image, where the source image is a real-time image acquired by an image collector.

[0059] When performing lane line detection, the image data of the driving environment, i.e., the source image, can be collected in real time through an image collector set in the vehicle. For example, the image collector can be a vehicle-mounted camera, and the lane line detection and obstacle detection can be performed through the real-time image collected by the vehicle-mounted camera.

[0060] Obstacle detection is the process of preprocessing and feature extraction of the collected data by sensors, and then analyzing it using machine learning or deep learning algorithms to identify and locate potential obstacles. For example, images captured by on-board cameras use image processing technology to extract obstacle features; while radar and lidar determine the distance and location of obstacles by transmitting and receiving signals.

[0061] The present application embodiment takes obstacle detection through image data as an example for explanation, and the image data is the source image described in the above embodiment. It should be noted that the obstacle detection in the present application embodiment is 3D obstacle detection, and the 3D obstacle detection method can be in various forms, which is not limited by the present application.

[0062] S2: Perform three-dimensional obstacle detection according to the source image to output an obstacle detection result.

[0063] After the source image is acquired, 3D obstacle detection is performed on the source image to output a corresponding 3D obstacle detection result, wherein the 3D obstacle detection result may include the three-dimensional coordinates of the obstacle, which are used to represent the position information of the obstacle in the three-dimensional space.

[0064] In some embodiments, when performing three-dimensional obstacle detection on a source image, the three-dimensional coordinates of obstacles in the source image can be detected by a BEV (Bird's Eye View) 3D detection network or a monocular vision 3D detection network to obtain the position information of the obstacles in the three-dimensional space. The three-dimensional coordinates can constitute a coordinate set, which can be used to describe the position information of the obstacles in the three-dimensional space.

[0065] S3: Analyze the grounding point of the obstacle in the obstacle detection result.

[0066] After obtaining the 3D obstacle detection result, the grounding point of the obstacle, that is, the contact point between the obstacle and the road surface, can be analyzed. Among them, the number of grounding points corresponding to a single obstacle can be multiple, so different obstacles correspond to different grounding point sets.

[0067] For example, 3D obstacle detection is performed on the source image collected in real time. After obtaining the 3D detection result of the obstacle, the grounding point in the obstacle detection result is determined, and the three-dimensional coordinates (x oi ,y oi ,z oi ), i = [0, N-1], N is the number of points.

[0068] S4: Constructing a pavement equation for fitting the pavement according to the grounding point.

[0069] After the grounding point of the obstacle is analyzed, the road surface equation of the fitted road surface is constructed according to the grounding point, where the fitted road surface is the corresponding road surface predicted according to the grounding point.

[0070] like Figure 2 As shown, in some embodiments, when constructing a road surface equation for fitting a road surface, the first target sub-coordinate of the ground contact point in the first direction can be read. Then, the first power form of the first target sub-coordinate is calculated according to a preset index. The preset index includes integers from 0 to 1, i.e., 0, 1, 2, and 3. Then, the first coefficient corresponding to the first power form is generated, and the first product of the first power form and the corresponding first coefficient is calculated. Then, the first sum of the first product is calculated, and the road surface equation is generated according to the first sum.

[0071] For example, construct the following slope equation z:

[0072]

[0073] In the formula, x o is the first target sub-coordinate, that is, the sub-coordinate in the x-axis direction; a r 、b r 、c r and d r is the first coefficient corresponding to different exponents.

[0074] In some embodiments, a first reference parameter is also generated according to the road surface equation. The first reference parameter is generated based on the first coefficient. A first objective function of the road surface equation is then constructed, wherein the goal of the first objective function is to minimize the sum of squares of the distances from the grounding point to the fitted road surface, and the first objective function includes the fitting weight of the grounding point and the first reference parameter. Then, the first objective function is solved based on an optimization algorithm (such as the optimization algorithm can be a gradient descent method, etc.) to obtain a value of the first reference parameter.

[0075] In some embodiments, the first objective function is as follows:

[0076]

[0077] In the formula, J r is the first objective function, ω i is the fitting weight of the grounding point, N is the number of grounding points, C r is the first reference parameter, λ r is a preset constant, z oi is the sub-coordinate of the ground point in a second direction, where the second direction is perpendicular to the first direction, equal x o is the sub-coordinate of the ground point in the first direction.

[0078] For example, the three-dimensional coordinates of the grounding point are (x oi ,y oi ,z oi ), i = [0, N-1], N is the number of points, then construct the following slope equation z:

[0079]

[0080] in:

[0081] C r =(a r ,b r ,c r ,d r );

[0082]

[0083] By building including C r The first objective function is to solve C r , to obtain the final road surface equation.

[0084] Accordingly, in some embodiments, the first objective function is as follows:

[0085]

[0086] In the formula, J r is the first objective function, ω i is the fitting weight of the grounding point, N is the number of grounding points, C r is the first reference parameter, λ r is a preset constant used to improve the speed and quality of optimization solution. oi is the sub-coordinate of the ground point in the second direction (z-axis direction), the second direction is perpendicular to the first direction, o is the sub-coordinate of the grounding point in the first direction (x-axis direction);

[0087] After constructing the road surface equation and the corresponding objective function, the objective function is solved, that is, the above C is calculated by the gradient descent method. r .

[0088] In some embodiments, the weight of the touchdown point can be set according to the position and confidence of the touchdown point relative to the vehicle. For example, a touchdown point far from the vehicle has a smaller weight than a close one, and an obstacle with a high confidence has a larger weight than an obstacle with a low confidence.

[0089] S5: Obtain a two-dimensional lane line detection result, where the two-dimensional lane line detection result is a detection result of performing two-dimensional lane line detection on the source image, and the two-dimensional lane line detection result includes pixel coordinates of the lane line.

[0090] While performing obstacle detection and constructing road surface equations through source images, 2D lane line detection can also be performed through source images to output corresponding two-dimensional lane line detection results. The process of performing 2D lane line detection on the source image and the process of performing 3D lane line detection on the source image can be performed simultaneously, or 3D obstacle detection can be performed based on the source image first and then 2D lane line detection, or 2D lane line detection can be performed based on the source image first and then 3D obstacle detection, which is not limited by this application.

[0091] In some embodiments, when performing two-dimensional lane line detection based on a source image, the source image can be input into a GANet (Global Association Network) to output a two-dimensional lane line detection result including lane line pixel points through GANet. For example, the pixel coordinates of the detected lane line are (u i , v i ).

[0092] S6: Calculate the simulated three-dimensional coordinates of the lane line based on the road surface equation and the two-dimensional lane line detection result.

[0093] After obtaining the two-dimensional lane line detection results, they can be converted into three-dimensional coordinates corresponding to the vehicle body coordinate system, that is, simulated three-dimensional coordinates. For example, the pixel coordinates of the lane line are (u i , v i ), the corresponding coordinate in the vehicle coordinate system is (x i ,y i ,z i ), i = [0, M-1], M is the number of lane line pixels.

[0094] like Figure 3 As shown, in some embodiments, when calculating the simulated three-dimensional coordinates of the lane line, the internal and external parameters of the camera associated with the image collector are read. Then, according to the internal and external parameters of the camera and the two-dimensional lane line detection results, the pixel coordinates of the lane line are converted into reference three-dimensional coordinates. Among them, the pixel coordinates include sub-coordinates in the first direction and the third direction, the first direction is perpendicular to the third direction, and the reference three-dimensional coordinates include unknown sub-coordinates of the pixel coordinates and the second direction, and the second direction is perpendicular to the first direction and the third direction, such as the first direction is the x-axis direction, the second direction is the z-axis direction, and the third direction is the y-axis direction. Then solve the unknown sub-coordinates according to the road surface equation to obtain the simulated three-dimensional coordinates of the lane line.

[0095] For example, combined with the road surface equation, the unknown sub-coordinates can be solved according to the following formula.

[0096]

[0097] like Figure 4 As shown, in some embodiments, after reading the internal and external parameters of the camera associated with the image collector, the corresponding internal parameter matrix and external parameter matrix can be analyzed according to the internal and external parameters of the camera. Then, the conversion equation of the reference three-dimensional coordinates is constructed through the internal parameter matrix, the external parameter matrix and the pixel point coordinates, and the reference three-dimensional coordinates are generated through the conversion equation.

[0098] In some embodiments, the internal parameter matrix includes:

[0099]

[0100] The external parameter matrix includes:

[0101]

[0102] The conversion equations of the base 3D coordinates include:

[0103]

[0104] Where s is the scale factor, u i is the sub-coordinate of the pixel point coordinate in the first direction, v i is the sub-coordinate of the pixel point coordinate in the third direction, f x and f y is the focal length of the camera’s internal and external parameters, c x and c y is the optical center of the camera's intrinsic parameters, R is the rotation matrix of the camera's intrinsic and extrinsic parameters, T is the translation vector of the camera's intrinsic and extrinsic parameters, (x i ,y i , z i ) is the reference three-dimensional coordinate.

[0105] For example, the pixel coordinates of the lane line are (u i , v i ), the corresponding coordinate in the vehicle coordinate system is (x i ,y i ,z i ), i = [0, M-1], M is the number of lane line pixels; the simulated three-dimensional coordinates (x i ,y i ,z i ):

[0106]

[0107] S7: Constructing a lane line equation according to the simulated three-dimensional coordinates.

[0108] After calculating the simulated three-dimensional coordinates, a lane line equation can be constructed according to the simulated three-dimensional coordinates, so as to convert the two-dimensional lane line detection result into the corresponding three-dimensional lane line detection result through the lane line equation.

[0109] Similarly, in some embodiments, the second target sub-coordinate of the reference three-dimensional coordinate in the first direction can be read. Then, the second power form of the second target sub-coordinate is calculated according to the preset index, where the preset index includes 0, 1, 2, and 3. Then, a second coefficient corresponding to the second power form is generated, and a second product of the second power form and the corresponding second coefficient is calculated. A second sum of the second products is calculated, and a lane line equation is generated according to the second sum.

[0110] For example, the lane line equation is as follows:

[0111]

[0112] It should be noted that the various parameters in the above formula can refer to the meanings of the various parameters in the slope equation, and this application will not elaborate on them here.

[0113] like Figure 5 As shown, in some embodiments, a second reference parameter is also generated according to the lane line equation. The second reference parameter is generated based on the second coefficient. A second objective function of the lane line equation is then constructed. The goal of the second objective function is to minimize the sum of squares of the distances from the simulated three-dimensional coordinates to the lane line equation, and the second objective function includes the fitting weights of the simulated three-dimensional coordinates and the second reference parameter. Then, the second objective function is solved based on an optimization algorithm to obtain the value of the second reference parameter.

[0114] For example, the lane line equation is as follows:

[0115]

[0116] The second objective function is:

[0117]

[0118] Then, by solving the second objective function, that is, solving the second reference parameter C through the optimization algorithm z and C y It should be noted that the various parameters in the above formula can refer to the meanings of the various parameters in the slope equation and the first objective function, and will not be elaborated in this application.

[0119] S8: Generate a three-dimensional lane line detection result using the lane line equation, wherein the three-dimensional lane line detection result includes the three-dimensional coordinates of the lane line.

[0120] After solving the lane line equation, the two-dimensional lane line detection result can be converted into a three-dimensional lane line detection result through the lane line equation to output the position information of the lane line in three-dimensional space, thereby reducing the complexity of lane line detection and achieving more accurate lane line detection.

[0121] Based on the above lane line detection method based on obstacle detection, some embodiments of the present application also provide a lane line detection system based on obstacle detection, which can be used to execute the above lane line detection method. Figure 6 As shown, the lane line detection system includes an input module 601, an obstacle perception module 602, a calculation module 603 and a lane line perception module 604, wherein:

[0122] The input module 601 is configured to obtain a source image, where the source image is a real-time image acquired by an image collector;

[0123] The obstacle sensing module 602 is configured to perform three-dimensional obstacle detection according to the source image to output an obstacle detection result, wherein the obstacle detection result includes the three-dimensional coordinates of the obstacle;

[0124] The calculation module 603 is configured to analyze the grounding point of the obstacle in the obstacle detection result; and construct a road surface equation for fitting the road surface according to the grounding point;

[0125] The lane line perception module 604 is configured to obtain a two-dimensional lane line detection result, which is a detection result of performing two-dimensional lane line detection on the source image, and the two-dimensional lane line detection result includes the pixel coordinates of the lane line; calculate the simulated three-dimensional coordinates of the lane line based on the road surface equation and the two-dimensional lane line detection result; construct a lane line equation according to the simulated three-dimensional coordinates; and generate a three-dimensional lane line detection result through the lane line equation, and the three-dimensional lane line detection result includes the three-dimensional coordinates of the lane line.

[0126] Based on the above lane line detection method and system, some embodiments of the present application also provide an electronic device 500, which may include the lane line detection system based on obstacle detection described in the above embodiment.

[0127] In some embodiments, Figure 7As shown, the electronic device 500 includes: at least one processor 501, at least one communication interface 502, at least one memory 503 and at least one bus 504. Among them, the bus 504 is used to realize the direct connection and communication of these components, the communication interface 502 is used to communicate signaling or data with other node devices, and the memory 503 stores computer program instructions that can be executed by the processor 501. When the electronic device 500 is running, the processor 501 communicates with the memory 503 through the bus 504, the processor 501 can call the computer program stored in the memory 503, and the execution of the computer program can realize the lane line detection method based on obstacle detection provided in the embodiment of the present application.

[0128] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0129] For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.

Claims

1. A lane line detection method based on obstacle detection, characterized in that: include: Acquire a source image, where the source image is a real-time image acquired by an image collector; Performing three-dimensional obstacle detection according to the source image to output an obstacle detection result, wherein the obstacle detection result includes the three-dimensional coordinates of the obstacle; parsing a grounding point of the obstacle in the obstacle detection result; Constructing a pavement equation for fitting the pavement according to the grounding point; Acquire a two-dimensional lane line detection result, where the two-dimensional lane line detection result is a detection result of performing two-dimensional lane line detection on the source image, and the two-dimensional lane line detection result includes pixel coordinates of the lane line; Calculating the simulated three-dimensional coordinates of the lane line based on the road surface equation and the two-dimensional lane line detection result; Constructing a lane line equation according to the simulated three-dimensional coordinates; A three-dimensional lane line detection result is generated by the lane line equation, and the three-dimensional lane line detection result includes the three-dimensional coordinates of the lane line.

2. The lane line detection method based on obstacle detection according to claim 1, characterized in that: The step of constructing a pavement equation for fitting the pavement according to the grounding point comprises: Reading a first target sub-coordinate of the touchdown point in a first direction; Calculate a first power form of the first target sub-coordinate according to a preset index, where the preset index includes 0, 1, 2, and 3; generating a first coefficient corresponding to the first power form; Calculating a first product of the first power form and the corresponding first coefficient; A first sum of the first products is calculated, and the road surface equation is generated according to the first sum.

3. The lane line detection method based on obstacle detection according to claim 2, characterized in that: After the step of constructing a pavement equation for fitting the pavement according to the grounding point, the method further includes: generating a first reference parameter according to the road surface equation, wherein the first reference parameter is generated based on the first coefficient; Constructing a first objective function of the road surface equation, wherein the first objective function aims to minimize the sum of squares of distances from the contact point to the fitted road surface, and the first objective function includes a fitting weight of the contact point and the first reference parameter; The first objective function is solved based on an optimization algorithm to obtain a value of the first reference parameter.

4. The lane line detection method based on obstacle detection according to claim 3, characterized in that: The first objective function is shown as follows: In the formula, J r is the first objective function, ω i is the fitting weight of the grounding point, N is the number of grounding points, C r is the first reference parameter, λ r is a preset constant, z oi is the sub-coordinate of the ground point in a second direction, where the second direction is perpendicular to the first direction, equal x o is the sub-coordinate of the ground point in the first direction.

5. The lane line detection method based on obstacle detection according to claim 1, characterized in that: The calculating the simulated three-dimensional coordinates of the lane line based on the road surface equation and the two-dimensional lane line detection result includes: Reading the internal and external parameters of the camera associated with the image collector; According to the camera internal and external parameters and the two-dimensional lane line detection result, the pixel point coordinates of the lane line are converted into reference three-dimensional coordinates; the pixel point coordinates include sub-coordinates in a first direction and a third direction, the first direction is perpendicular to the third direction, and the reference three-dimensional coordinates include the pixel point coordinates and unknown sub-coordinates in a second direction, the second direction is perpendicular to the first direction and the third direction; The unknown sub-coordinates are solved according to the road surface equation to obtain the simulated three-dimensional coordinates of the lane line.

6. The lane line detection method based on obstacle detection according to claim 5, characterized in that: After the step of reading the internal and external parameters of the camera associated with the image collector, the method further includes: Analyze the corresponding intrinsic parameter matrix and extrinsic parameter matrix according to the intrinsic and extrinsic parameters of the camera; Constructing a conversion equation of the reference three-dimensional coordinates through the intrinsic parameter matrix, the extrinsic parameter matrix and the pixel point coordinates; The reference three-dimensional coordinates are generated by the conversion equation.

7. The lane line detection method based on obstacle detection according to claim 5, characterized in that: The constructing a lane line equation according to the simulated three-dimensional coordinates includes: Reading a second target sub-coordinate of the reference three-dimensional coordinate in the first direction; Calculate the second power form of the second target sub-coordinate according to a preset index, where the preset index includes 0, 1, 2, and 3; generating a second coefficient corresponding to the second power form; calculating a second product of the second power form and the corresponding second coefficient; Calculate a second sum of the second products, and generate the lane line equation according to the second sum.

8. The lane line detection method based on obstacle detection according to claim 7, characterized in that: After the step of constructing the lane line equation according to the simulated three-dimensional coordinates, the method further includes: generating a second reference parameter according to the lane line equation, wherein the second reference parameter is generated based on the second coefficient; Constructing a second objective function of the lane line equation, wherein the goal of the second objective function is to minimize the sum of squares of distances from the simulated three-dimensional coordinates to the lane line equation, and the second objective function includes a fitting weight of the simulated three-dimensional coordinates and the second reference parameter; The second objective function is solved based on an optimization algorithm to obtain a value of the second reference parameter.

9. A lane line detection system based on obstacle detection, characterized in that: include: An input module is configured to acquire a source image, wherein the source image is a real-time image acquired by an image collector; an obstacle sensing module, configured to perform three-dimensional obstacle detection according to the source image to output an obstacle detection result, wherein the obstacle detection result includes the three-dimensional coordinates of the obstacle; A calculation module, configured to analyze a grounding point of the obstacle in the obstacle detection result; Constructing a pavement equation for fitting the pavement according to the grounding point; A lane line sensing module is configured to obtain a two-dimensional lane line detection result, wherein the two-dimensional lane line detection result is a detection result of performing two-dimensional lane line detection on the source image, and the two-dimensional lane line detection result includes pixel coordinates of the lane line; Calculating the simulated three-dimensional coordinates of the lane line based on the road surface equation and the two-dimensional lane line detection result; Constructing a lane line equation according to the simulated three-dimensional coordinates; A three-dimensional lane line detection result is generated by the lane line equation, and the three-dimensional lane line detection result includes the three-dimensional coordinates of the lane line.

10. An electronic device, characterized in that: Includes processor, memory and bus; The processor and the memory are communicatively connected via the bus; The memory stores computer program instructions executable by the processor, the processor being configured to: Acquire a source image, where the source image is a real-time image acquired by an image collector; Performing three-dimensional obstacle detection according to the source image to output an obstacle detection result, wherein the obstacle detection result includes the three-dimensional coordinates of the obstacle; parsing a grounding point of the obstacle in the obstacle detection result; Constructing a pavement equation for fitting the pavement according to the grounding point; Acquire a two-dimensional lane line detection result, where the two-dimensional lane line detection result is a detection result of performing two-dimensional lane line detection on the source image, and the two-dimensional lane line detection result includes pixel coordinates of the lane line; Calculating the simulated three-dimensional coordinates of the lane line based on the road surface equation and the two-dimensional lane line detection result; Constructing a lane line equation according to the simulated three-dimensional coordinates; A three-dimensional lane line detection result is generated by the lane line equation, and the three-dimensional lane line detection result includes the three-dimensional coordinates of the lane line.

Images