A road boundary line detection method, device, vehicle and electronic device

Abstract

The application relates to a road boundary line detection method and device, a vehicle and electronic equipment, and relates to the technical field of vehicles, in particular to the technical field of intelligent driving. The method comprises the following steps: determining a reference boundary line; the reference boundary line comprises a radar detection boundary line or a forward-looking detection boundary line; screening candidate fitting points from radar detection points based on the reference boundary line; the transverse distance between the candidate fitting points and the reference boundary line is not more than a distance threshold; performing curve fitting based on the candidate fitting points to obtain a boundary fitting line; and fusing the reference boundary line and the boundary fitting line to obtain a road boundary line. Through the optimization of the multi-sensor fusion technology and the environmental perception algorithm, the robustness and reliability of the intelligent driving system in a curved road environment are significantly improved, and the safe and stable operation of the vehicle is ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicles, in particular to the technical field of intelligent driving, and specifically to a road boundary line detection method and device, a vehicle and an electronic device. BACKGROUND

[0002] With the rapid development of artificial intelligence technology, intelligent driving technology is accelerating iteration and entering the practical stage. Intelligent driving technology realizes the control of vehicle acceleration, deceleration and steering through real-time perception of surrounding environment information of the vehicle, not only effectively avoids traffic problems caused by driver errors, improves driving safety, but also reduces the work load of the driver and improves driving comfort.

[0003] Intelligent driving technology mainly includes two parts: surrounding environment perception and self-vehicle function control. As the basis for self-vehicle function control, surrounding environment perception is a crucial task. In perception, the detection of road boundary lines is crucial to ensure driving safety. Accurate and stable road boundary line detection helps to assist in designing self-vehicle function logic, avoiding vehicle deviation from the road or collision accidents, thereby improving driving safety.

[0004] In related technologies, there is a method of detecting road boundary lines based on a forward-looking camera. For example, a road boundary line detection method based on a vehicle-mounted millimeter wave radar discloses: obtaining radar point cloud information and vehicle body information of a current road scene; filtering out moving point clouds through dynamic and static separation operation according to the obtained target absolute speed; combining the motion state of the current vehicle, and performing road scene recognition by applying different point filtering and clustering strategies. The technical solution cannot effectively identify road edge points, and can detect relatively accurate road boundary lines in a good road condition, but the accuracy and stability of the detected road boundary lines sharply decrease in a poor road condition, affecting driving safety. SUMMARY

[0005] The present application provides a road boundary line detection method, device, vehicle and electronic device to significantly improve the robustness and reliability of the intelligent driving system in a curved road environment and ensure safe and stable operation of the vehicle.

[0006] According to a first aspect of the present application, a road boundary line detection method is provided, the method comprising:

[0007] determining a reference boundary line; the reference boundary line includes a radar detection boundary line or a forward-looking detection boundary line;

[0008] screening candidate matching points from radar detection points based on the reference boundary line; the transverse distance between the candidate matching points and the reference boundary line does not exceed a distance threshold;

[0009] perform curve fitting based on the candidate fitting points to obtain a boundary fitting line;

[0010] fuse the reference boundary line and the boundary fitting line to obtain a road boundary line.

[0011] It can be seen that, by applying the embodiment scheme, the reference boundary line is first determined, the reference boundary line can include a radar detection boundary line or a forward-looking detection boundary line, candidate fitting points are selected from radar detection points based on the reference boundary line, then curve fitting is performed based on the candidate fitting points to obtain a boundary fitting line. The reference boundary line and the boundary fitting line are fused to obtain a road boundary line. The road detection characteristics of the forward-looking camera and the radar are comprehensively considered and fused. In different scenarios, such as straight road or curved road scenarios, the fusion mode can be reasonably adapted to improve the robustness of road boundary line detection. Especially for high-speed, urban expressway, ramp and other curved road scenarios with guardrails, through the optimization of multi-sensor fusion technology and environmental perception algorithm, the robustness and reliability of the intelligent driving system in the curved road environment are significantly improved, ensuring the safe and stable operation of the vehicle.

[0012] In a possible manner, the distance threshold is determined by superimposing a correction value on a preset lateral distance; the correction value is determined based on the lateral distance between the candidate fitting point selected at the historical moment and the reference boundary line.

[0013] In a possible manner, the radar detection points include radar detection basic points; the method further includes:

[0014] Based on the direction trend of the extension of the forward-looking detection boundary line, the radar detection basic points are taken as a reference to perform interval sampling to obtain interval sampling points;

[0015] The interval sampling points are added to the radar detection points.

[0016] Since the interval sampling points are sampled based on the extension direction of the forward-looking detection boundary line, the characteristic information of the forward-looking detection boundary line is indirectly utilized, realizing the joint processing of the forward-looking detection boundary line and the radar detection boundary line, and at the same time solving the technical problem that the sparsity of the radar detection points affects the detection accuracy.

[0017] In a possible manner, the fusion of the reference boundary line and the boundary fitting line to obtain a road boundary line includes:

[0018] fuse the reference boundary line and the boundary fitting line to obtain a fusion result;

[0019] perform filtering processing on the latest road boundary line as a state quantity and the fusion result as an observation quantity to obtain an updated road boundary line.

[0020] It can be seen that, by using the Kalman filtering method, the characteristics of the reference boundary line and the boundary fitting line can be integrated, the sensor noise can be effectively filtered out, and even if the sensor data of a frame is lost, a smooth and stable road boundary line can be output.

[0021] In a possible manner, the fusing of the reference boundary line and the boundary fitting line comprises:

[0022] determining a first fusion weight corresponding to the reference boundary line and a second fusion weight corresponding to the boundary fitting line based on a curvature of the reference boundary line or the boundary fitting line;

[0023] fusing the reference boundary line and the boundary fitting line based on the first fusion weight and the second fusion weight.

[0024] In a possible manner, the method further comprises:

[0025] determining the first fusion weight and the second fusion weight based on a size relationship between the curvature and a preset curvature;

[0026] wherein, when the curvature is greater than the preset curvature, the first fusion weight is less than the second fusion weight; and when the curvature is less than the preset curvature, the first fusion weight is greater than the second fusion weight.

[0027] Thus, in a straight road scene, more reliance is placed on the detection accuracy of the radar-detected boundary line, and a greater fusion weight is allocated; in a curved road scene, the detection accuracy of the radar-detected boundary line may not meet the requirements, while the determination of the boundary fitting line takes into account the lateral distance between the radar detection points and the reference boundary line, and the lateral distance threshold is dynamically adjusted in the curved road scene, so that the boundary fitting line can better reflect the real road boundary line in the curved road scene, and a greater fusion weight is allocated. By adaptively adjusting the fusion weight, the finally determined road boundary line has good performance in both straight and curved road scenes.

[0028] In a possible manner, the method further comprises:

[0029] performing an abnormality check on the road boundary line based on the number of inliers and / or outliers in the radar detection points;

[0030] Alternatively, performing an abnormality check on the road boundary line based on a cumulative number of stable frames of the road boundary line;

[0031] Alternatively, performing an abnormality check on the road boundary line based on a difference between a parameter of the visual lane line and a parameter of the road boundary line; wherein the parameter comprises at least one of a parallel state, a slope, and a curvature.

[0032] It can be seen that after determining the road boundary line, various methods are used for anomaly checking, so as to maximize the output of correct road boundary lines and avoid output of incorrect road boundary lines as much as possible.

[0033] In a possible manner, the determining the reference boundary line comprises:

[0034] It is determined whether the determined road boundary line exists at present, and if yes, the determined road boundary line is taken as the reference boundary line.

[0035] If no, in a case where the transverse distance restriction condition is not met, the radar detection boundary line is selected as the reference boundary line; and in a case where the transverse distance restriction condition is met, the forward-looking detection boundary line is selected as the reference boundary line; the transverse distance restriction condition represents that the transverse distance of the forward-looking detection boundary line is less than the transverse distance of the radar detection boundary line.

[0036] According to a second aspect provided in the present application, a road boundary line detection device is provided, and the device comprises:

[0037] A determining module is configured to determine a reference boundary line; the reference boundary line comprises a radar detection boundary line or a forward-looking detection boundary line.

[0038] A screening module is configured to screen a candidate fitting point from radar detection points based on the reference boundary line; the transverse distance between the candidate fitting point and the reference boundary line is not more than a preset distance threshold.

[0039] A fitting module is configured to perform curve fitting based on the candidate fitting point to obtain a boundary fitting line.

[0040] A fusion module is configured to fuse the reference boundary line and the boundary fitting line to obtain a road boundary line.

[0041] According to the embodiments of the present application, the reference boundary line is first determined, the reference boundary line can comprise a radar detection boundary line or a forward-looking detection boundary line, a candidate fitting point is screened from radar detection points based on the reference boundary line, then curve fitting is performed based on the candidate fitting point to obtain a boundary fitting line. The reference boundary line and the boundary fitting line are fused to obtain a road boundary line. The road detection features of the forward-looking camera and the radar are comprehensively considered and fused. In different scenarios, such as straight road or curved road scenarios, the fusion manner can be reasonably adapted to improve the robustness of road boundary line detection. Especially for high-speed, urban expressway, ramp and other curved road scenarios with guardrails, through optimization of multi-sensor fusion technology and environmental perception algorithm, the robustness and reliability of the intelligent driving system in the curved road environment are significantly improved, and the safe and stable operation of the vehicle is ensured.

[0042] In one possible approach, the distance threshold is determined by adding a correction value to a preset lateral distance; the correction value is determined based on the lateral distance between the candidate fitting point selected at a historical time and the baseline boundary line.

[0043] In one possible embodiment, the radar detection point includes a radar detection base point; the device further includes:

[0044] A module is used to perform interval sampling based on the directional trend of the forward-looking detection boundary line extension, with the radar detection base point as a reference, to obtain interval sampling points; and the interval sampling points are added to the radar detection points.

[0045] In one possible approach, the fusion module is specifically used to: fuse the baseline boundary line and the boundary fitting line to obtain a fusion result; and use the latest road boundary line as a state variable and the fusion result as an observation to perform filtering processing to obtain an updated road boundary line.

[0046] In one possible approach, the fusion module is specifically configured to: determine a first fusion weight corresponding to the reference boundary line and a second fusion weight corresponding to the boundary fitting line based on the curvature of the reference boundary line or the boundary fitting line; and fuse the reference boundary line and the boundary fitting line based on the first fusion weight and the second fusion weight.

[0047] In one possible approach, the determining module is further configured to: determine the first fusion weight and the second fusion weight based on the relationship between the curvature and the preset curvature; wherein, when the curvature is greater than the preset curvature, the first fusion weight is less than the second fusion weight; and when the curvature is less than the preset curvature, the first fusion weight is greater than the second fusion weight.

[0048] In one possible embodiment, the device further includes a verification module for: performing anomaly verification on the road boundary line based on the number of inner and / or outer points in the radar detection points; or, performing anomaly verification on the road boundary line based on the cumulative number of stable frames of the road boundary line; or, performing anomaly verification on the road boundary line based on the degree of difference between the parameters of the visual lane line and the parameters of the road boundary line; wherein the parameters include at least one of parallelism, slope, and curvature.

[0049] In one possible approach, a module is identified, specifically for:

[0050] Determine whether there is a defined road boundary line. If so, use the defined road boundary line as the reference boundary line.

[0051] If not, the radar detection boundary line is selected as the reference boundary line if the lateral distance constraint is not met; if the lateral distance constraint is met, the forward-looking detection boundary line is selected as the reference boundary line; the lateral distance constraint indicates that the lateral distance of the forward-looking detection boundary line is less than the lateral distance of the radar detection boundary line.

[0052] According to a third aspect provided in this application, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute instructions to implement the method of the first aspect described above and any possible implementation thereof.

[0053] According to a fourth aspect provided in this application, a computer-readable storage medium is provided that, when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to perform the methods described in the first aspect and any possible implementation thereof.

[0054] According to the fifth aspect provided in this application, a computer program product is provided, the computer program product including computer instructions, which, when executed on an electronic device, cause the electronic device to perform the method described in the first aspect and any possible implementation thereof.

[0055] According to the sixth aspect provided in this application, a vehicle is provided that includes the road boundary detection device described in the second aspect above.

[0056] It should be noted that the technical effects of any of the implementation methods in aspects two through six can be found in the technical effects of the corresponding implementation methods in aspect one, and will not be repeated here.

[0057] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0058] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application, and do not constitute an undue limitation of this application.

[0059] Figure 1 This is a flowchart illustrating a road boundary line detection method according to an exemplary embodiment;

[0060] Figure 2 This is a flowchart illustrating a fusion method according to an exemplary embodiment;

[0061] Figure 3 This is a block diagram illustrating a road boundary detection device according to an exemplary embodiment;

[0062] Figure 4 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Detailed Implementation

[0063] To enable those skilled in the art to better understand the technical solutions of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0064] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0065] In the embodiments of this application, the words "exemplary," "for example," or "for instance" are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the words "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a specific manner.

[0066] Currently, existing methods for road boundary detection based on forward-looking cameras experience a sharp decline in accuracy and stability in scenarios such as curves and slopes. Another approach, using LiDAR for road boundary detection, is more expensive.

[0067] In view of this, this application provides a road boundary line detection method, apparatus, system, and vehicle. It employs a collaborative approach using forward-looking and millimeter-wave radar. The forward-looking radar detects the boundary line to select radar detection points, then fits the selected radar detection points to obtain a boundary fitting line, which is then fused with a reference boundary line to obtain an accurate and stable road boundary line. Furthermore, the embodiments of this application use millimeter-wave radar for road boundary line detection, avoiding the increased cost associated with using lidar.

[0068] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0069] In this application embodiment, a vehicle may also be referred to as a vehicle, mobile carrier, electric vehicle (EV), hybrid electric vehicle (HEV), plug-in hybrid electric vehicle (PHEV), fuel cell vehicle (FCV), autonomous vehicle, intelligent and connected vehicle (ICV), driverless vehicle, etc.

[0070] In this application, the vehicle can be a sedan, a sport utility vehicle (SUV), a truck, an electric vehicle, a motorcycle, a tricycle, a special vehicle (such as an ambulance, fire truck, police car, etc.), a driverless taxi, an intelligent connected bus, an autonomous logistics vehicle, an electric truck, etc. Furthermore, this method is also applicable to various special-purpose vehicles, such as agricultural vehicles, mining vehicles, forestry vehicles, airport vehicles, and port vehicles. This application does not impose specific limitations in this regard.

[0071] For ease of understanding, the road boundary line detection method provided in this application is described in detail below with reference to the accompanying drawings. See also... Figure 1 The method may include the following steps:

[0072] S101: Determine the baseline boundary line; the baseline boundary line includes the radar detection boundary line.

[0073] The road boundary detection method provided in this application can be applied to a control unit in a vehicle. Specifically, the control unit can be a domain controller or an ECU. The control unit receives sensor data via Ethernet or CAN signals, which may specifically include forward-looking camera data and forward radar data.

[0074] For forward-looking radar data, radar detection points can be verified, which can be done in two ways. First, verification can be performed by combining forward-looking camera data with the radar detection points. For example, a longitudinal distance threshold can be set. For radar detection points whose longitudinal distance exceeds the threshold (e.g., 60m), if a corresponding visual detection point exists, the radar detection point is considered correctly identified. Second, if a radar detection point is stably detected for multiple consecutive frames (e.g., 3 frames), the radar detection point is considered to have passed verification.

[0075] In this embodiment of the application, after the radar detection points are verified, the baseline boundary line is first determined, and the baseline boundary line serves as the basis for selecting radar detection points.

[0076] In one embodiment of this application, the reference boundary line can be either a radar detection boundary line or a forward-looking detection boundary line. The radar detection boundary line represents the road boundary line initially determined based on radar data, also known as a radar guardrail line; the forward-looking detection boundary line represents the road boundary line initially determined based on forward-looking camera data, also known as a forward-looking roadside line.

[0077] In one embodiment of this application, a priority can be set to reasonably select the baseline boundary line under different circumstances.

[0078] For example, firstly, it is determined whether a defined road boundary line exists. If so, the defined road boundary line is used as the reference boundary line. The defined road boundary line is determined using the road boundary line detection method provided in this application embodiment. This method is continuously executed; if a defined road boundary line exists, the road boundary line determined in the previous frame is preferentially used as the reference boundary line for the current frame.

[0079] If no defined road boundary line exists, and the lateral distance constraint is not met, the radar-detected boundary line is selected as the reference boundary line. If the lateral distance constraint is met, the forward-looking detection boundary line is selected as the reference boundary line; the lateral distance constraint indicates that the lateral distance of the forward-looking detection boundary line is less than the lateral distance of the radar-detected boundary line.

[0080] For example, the radar detection boundary line has a higher priority than the forward-looking detection boundary line, that is, the radar detection boundary line is preferred as the reference boundary line. Furthermore, by adjusting the lateral distance, the priority of the radar detection boundary line and the forward-looking detection boundary line is adjusted, and the detection boundary line with a closer lateral distance is more likely to be selected as the reference boundary line.

[0081] For example, if the horizontal distance of the radar detection boundary line is greater than 4.5 meters and the horizontal distance of the forward-looking detection boundary line is less than 3.0 meters, then the forward-looking detection boundary line can be selected as the reference boundary line.

[0082] Furthermore, if there is no radar detection boundary line in the previous frame, the forward-looking detection boundary line can be selected as the reference boundary line.

[0083] S102: Select candidate fitting points from radar detection points based on the baseline boundary line; the lateral distance between the candidate fitting point and the baseline boundary line does not exceed the distance threshold.

[0084] In this embodiment, the baseline boundary line is used as the basis for selecting candidate fitting points, that is, the radar detection points that are laterally close enough to the baseline boundary line are selected from the radar detection points for subsequent curve fitting.

[0085] For example, along the baseline boundary line, each radar detection point is traversed from near to far. The lateral distance from the radar detection point to the baseline boundary line is calculated, and the relationship between the lateral distance and a distance threshold is determined. If the lateral distance is less than the distance threshold, the radar detection point is considered a point on the road boundary and is identified as a candidate fitting point, which needs to participate in curve fitting. The distance threshold can be set according to actual needs, for example, 3 meters.

[0086] In one embodiment of this application, the distance threshold is determined by adding a correction value to a preset lateral distance, wherein the correction value is determined based on the lateral distance between the candidate fitting point selected at a historical time and the baseline boundary line.

[0087] For example, during the process of traversing radar detection points, for the selected historical candidate fitting points, a correction value is determined based on the statistical lateral distance of the candidate fitting points to compensate for the preset lateral distance, thereby updating the distance threshold in real time.

[0088] For example, in straight-line conditions, radar detection points are relatively clustered, so the lateral distances corresponding to the selected historical candidate fitting points are usually small. For instance, if the preset lateral distance is 3 meters, and the historical candidate fitting points remain around 2 meters, a correction value can be determined based on the difference between the average lateral distance of the historical candidate fitting points and the preset lateral distance. If the determined correction value is -1 meter, then the current distance threshold is updated to 3 - 1 = 2 meters.

[0089] In curved road conditions, radar detection points are relatively evenly distributed on both sides of the baseline boundary line. Based on a distance threshold, radar detection points on either side of the baseline boundary line can be selected as candidate fitting points. For example, if the distance threshold is 3m, radar detection points within 3m on each side of the baseline boundary line can be selected as candidate fitting points. If positive and negative values ​​are used to represent the coordinates of the left and right sides, radar detection points with coordinate intervals of [-3m, +3m] can be determined as candidate fitting points.

[0090] In the case of curves, the density of radar detection points distributed on the left and right sides of the reference boundary line may differ. For example, there are more radar detection points distributed on the left side of the reference boundary line than on the right side. The number of candidate fitting points selected on the left side of the reference boundary line is also more than that on the right side. This can be represented by positive and negative values, that is, there are more candidate fitting points with negative coordinates than candidate fitting coordinates with positive coordinates.

[0091] In this embodiment, prediction can be made based on the historical distribution of candidate fitting points' coordinates. The prediction result is then used as a correction value, which is added to the initial coordinate interval to obtain the corrected coordinate interval. For example, prediction can be performed in real time using algorithms such as support vector machines or linear regression to obtain the prediction result.

[0092] For example, if the predicted value is -1 after predicting the coordinates of N candidate fitting points selected from historical moments, then the initial coordinate interval [-3m, +3m] is corrected to obtain the corrected coordinate interval [-4m, +2m]. Based on the corrected coordinate interval, radar detection points are selected, that is, radar detection points located to the left of the baseline boundary line with a lateral distance of less than 4m and located to the right of the baseline boundary line with a lateral distance of less than 2m are selected as candidate fitting points.

[0093] Therefore, the correction value is dynamically adjusted based on the lateral distance between the candidate fitting points selected at historical moments and the baseline boundary line. This allows the strategy for selecting candidate fitting points to be dynamically adjusted in scenarios where the road gradually transitions from a straight road to a curve, or vice versa, achieving the goal of adaptively selecting fitting candidate points in both straight and curved road scenarios.

[0094] S103: Perform curve fitting based on candidate fitting points to obtain the boundary fitting line.

[0095] In this embodiment, the algorithm used for curve fitting is not limited. For example, based on the selected candidate fitting points, the least squares method is used to perform curve fitting to obtain the boundary fitting line.

[0096] S104: Merge the baseline boundary line and the fitted boundary line to obtain the road boundary line.

[0097] The fusion performed in this step is a decision-level fusion, which can employ a decision-level fusion algorithm to ensure that the fused road boundary line incorporates the features of the baseline boundary line and the boundary fitting line, thus better reflecting the true road boundary.

[0098] As can be seen, applying the scheme of this application, a baseline boundary line is first determined. The baseline boundary line may include a radar detection boundary line or a forward-looking detection boundary line. Candidate fitting points are selected from radar detection points based on the baseline boundary line, and then curve fitting is performed based on the candidate fitting points to obtain the boundary fitting line. The baseline boundary line and the boundary fitting line are then fused to obtain the road boundary line. This comprehensively considers and fuses the road detection features of the forward-looking camera and radar. In different scenarios, such as straight roads or curves, the fusion method can be reasonably adapted to improve the robustness of road boundary line detection. Especially for curve scenarios with guardrails, such as highways, urban expressways, and ramps, the optimization of multi-sensor fusion technology and environmental perception algorithms significantly improves the robustness and reliability of the intelligent driving system in curve environments, ensuring the safe and stable operation of the vehicle.

[0099] In one embodiment of this application, the reference boundary line and the boundary fitting line are fused to obtain the road boundary line. Specifically, this may include: fusing the reference boundary line and the boundary fitting line to obtain a fusion result; using the latest road boundary line as the state variable and the fusion result as the observation, performing filtering processing to obtain the updated road boundary line.

[0100] For example, see Figure 2 A weighted average method can be used to fuse the point sets contained in the baseline boundary line and the fitted boundary line to obtain a preliminary fusion result. The baseline boundary line and the fitted boundary line correspond to the first fusion weight and the second fusion weight, respectively. Further, the fusion result is used as an observation, and the current road boundary line (i.e., the road boundary line of the previous frame) is used as a prediction, to perform Kalman filtering to obtain the road boundary line of the current frame. Kalman filtering can be considered an optimal recursive data processing algorithm. The core processing steps include prediction and correction. Prediction refers to predicting the current state based on the previous state of the system, and correction refers to obtaining the current observations and then combining the prediction and the observations to obtain the current optimal estimate.

[0101] As can be seen, the Kalman filter can integrate the features of the baseline boundary line and the boundary fitting line, effectively filter out sensor noise, and output a smooth and stable road boundary line even if sensor data of a certain frame is lost.

[0102] In one embodiment of this application, when fusing the baseline boundary line and the boundary fitting line, the fusion can be performed based on the curvature of the baseline boundary line or the boundary fitting line and their respective fusion weights.

[0103] For example, based on the curvature of the baseline boundary line or the boundary fitting line, it can be determined whether the current scene is biased towards a straight road scene or a curved road scene, and then an appropriate fusion weight can be selected.

[0104] For example, if a curvature threshold of 0.005 is set, and the curvature of either the baseline boundary line or the fitted boundary line exceeds the curvature threshold, then the current scene is considered to be a curved scene, and the first fusion weight is determined to be less than the second fusion weight. If the curvature of neither the baseline boundary line nor the fitted boundary line exceeds the curvature threshold, then the current scene is considered to be a straight scene, and the first fusion weight is determined to be greater than the second fusion weight.

[0105] Therefore, in straight-line scenarios, the detection accuracy of the baseline boundary line is more trusted, and a larger fusion weight is assigned. In curved scenarios, the detection accuracy of the baseline boundary line may not meet the requirements. The determination of the boundary fitting line considers the lateral distance between the radar detection point and the baseline boundary line, and the lateral distance threshold is dynamically adjusted in curved scenarios. Therefore, the boundary fitting line in curved scenarios better reflects the real road boundary line, and thus a larger fusion weight is assigned. By adaptively adjusting the fusion weight, the final determined road boundary line has good performance in both straight and curved scenarios.

[0106] In one embodiment of this application, the radar detection point includes a radar detection baseline point and interval sampling points obtained based on sampling. Specifically, the interval sampling points are determined as follows: based on the directional trend of the forward-looking detection boundary line, interval sampling is performed with the radar detection baseline point as a reference to obtain the interval sampling points. For example, along the trend direction of the forward-looking detection boundary line, equidistant sampling is performed every 10m to supplement the radar detection point with the interval sampling points.

[0107] Since the interval sampling points are sampled based on the extension direction of the forward-looking detection boundary line, the feature information of the forward-looking detection boundary line is indirectly utilized to realize the joint processing of the forward-looking detection boundary line and the radar detection boundary line. At the same time, it solves the technical problem that the sparse radar detection points affect the detection accuracy.

[0108] In one embodiment of this application, after determining the road boundary line of the current frame, output verification can be performed to suppress possible erroneous output. Specifically, this can include: performing anomaly verification on the road boundary line based on the number of inner and / or outer points in the radar detection points; or, performing anomaly verification on the road boundary line based on the cumulative number of stable frames of the road boundary line; or, performing anomaly verification on the road boundary line based on the degree of difference between the parameters of the visual lane line and the parameters of the road boundary line; wherein the parameters include at least one of parallelism, slope, and curvature.

[0109] For example, the first anomaly detection method involves traversing the radar detection points and determining the horizontal distance between each radar detection point and the road boundary line. The point is then classified as either an inner or outer point based on this horizontal distance. For instance, a horizontal distance less than 1.5 meters is considered an inner point, while a horizontal distance between 1.5 and 3 meters is considered an outer point. If the number of inner points is low or the number of outer points is high, the probability of false detections along the road boundary line is greater, thus suppressing the output. For example, if the number of inner points is less than 10 or the number of outer points is greater than 5, then a false detection of the road boundary line is determined.

[0110] The second type of anomaly detection uses a multi-frame confidence accumulation method. The more stable detection frames, the greater the confidence and the more likely it is to be output.

[0111] The third anomaly detection method combines the features of the visual lane lines. A forward-facing camera captures images of the lane lines, and the parametric features of the visual lane lines are analyzed, such as parallelism, slope, and curvature. If the parametric features of the determined road boundary lines differ significantly from those of the visual lane lines, the probability of false detection of the road boundary lines is high. In cases where the parametric features of the two differ significantly, the output of the road boundary lines is suppressed.

[0112] As can be seen, after determining the road boundary line, multiple methods are used for anomaly detection to maximize the output of the correct road boundary line while minimizing the output of the incorrect road boundary line.

[0113] The above primarily describes the solutions provided by the embodiments of this application from a methodological perspective. To achieve the above functions, the road boundary detection device or electronic device includes corresponding hardware structures and / or software modules for performing each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0114] According to the above method, the exemplary road boundary line detection device or electronic device can be divided into functional modules. For example, the road boundary line detection device or electronic device may include functional modules corresponding to each functional division, or two or more functions may be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0115] Figure 3 This is a block diagram illustrating a road boundary detection device according to an exemplary embodiment. (Refer to...) Figure 3 The device includes:

[0116] The determination module 301 is used to determine the reference boundary line; the reference boundary line includes the radar detection boundary line or the forward-looking detection boundary line.

[0117] The filtering module 302 is used to filter candidate fitting points from radar detection points based on the reference boundary line; the lateral distance between the candidate fitting point and the reference boundary line does not exceed a preset distance threshold.

[0118] Fitting module 303 is used to perform curve fitting based on the candidate fitting points to obtain the boundary fitting line;

[0119] The fusion module 304 is used to fuse the baseline boundary line and the boundary fitting line to obtain the road boundary line.

[0120] In one possible approach, the distance threshold is determined by adding a correction value to a preset lateral distance; the correction value is determined based on the lateral distance between the candidate fitting point selected at a historical time and the baseline boundary line.

[0121] In one possible embodiment, the radar detection point includes a radar detection base point; the device further includes:

[0122] A module is used to perform interval sampling based on the directional trend of the forward-looking detection boundary line extension, with the radar detection base point as a reference, to obtain interval sampling points; and the interval sampling points are added to the radar detection points.

[0123] In one possible approach, the fusion module is specifically used to: fuse the baseline boundary line and the boundary fitting line to obtain a fusion result; and use the latest road boundary line as a state variable and the fusion result as an observation to perform filtering processing to obtain an updated road boundary line.

[0124] In one possible approach, the fusion module is specifically configured to: determine a first fusion weight corresponding to the reference boundary line and a second fusion weight corresponding to the boundary fitting line based on the curvature of the reference boundary line or the boundary fitting line; and fuse the reference boundary line and the boundary fitting line based on the first fusion weight and the second fusion weight.

[0125] In one possible approach, the determining module is further configured to: determine the first fusion weight and the second fusion weight based on the relationship between the curvature and the preset curvature; wherein, when the curvature is greater than the preset curvature, the first fusion weight is less than the second fusion weight; and when the curvature is less than the preset curvature, the first fusion weight is greater than the second fusion weight.

[0126] In one possible embodiment, the device further includes a verification module for: performing anomaly verification on the road boundary line based on the number of inner and / or outer points in the radar detection points; or, performing anomaly verification on the road boundary line based on the cumulative number of stable frames of the road boundary line; or, performing anomaly verification on the road boundary line based on the degree of difference between the parameters of the visual lane line and the parameters of the road boundary line; wherein the parameters include at least one of parallelism, slope, and curvature.

[0127] In one possible approach, a module is identified, specifically for:

[0128] Determine whether there is a defined road boundary line. If so, use the defined road boundary line as the reference boundary line.

[0129] If not, the radar detection boundary line is selected as the reference boundary line if the lateral distance constraint is not met; if the lateral distance constraint is met, the forward-looking detection boundary line is selected as the reference boundary line; the lateral distance constraint indicates that the lateral distance of the forward-looking detection boundary line is less than the lateral distance of the radar detection boundary line.

[0130] Figure 4 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Figure 4 As shown, the electronic device includes, but is not limited to, a processor 401 and a memory 402.

[0131] The memory 402 described above is used to store the executable instructions of the processor 401. It is understood that the processor 401 is configured to execute instructions to implement the road boundary detection method in the above embodiments.

[0132] It should be noted that those skilled in the art will understand that Figure 4 The electronic device structure shown does not constitute a limitation on the electronic device; the electronic device may include, but is not limited to, other electronic devices. Figure 4 This may indicate more or fewer components, or combinations of certain components, or different component arrangements.

[0133] Processor 401 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines. By running or executing software programs and / or modules stored in memory 402, and by calling data stored in memory 402, it performs various functions and processes data, thereby providing overall monitoring of the electronic device. Processor 401 may include one or more processing units. Optionally, processor 401 may integrate an application processor and a modem processor. The application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into processor 401.

[0134] The memory 402 can be used to store software programs and various data. The memory 402 may primarily include a program storage area and a data storage area. The program storage area may store the operating system, application programs required by at least one functional module (such as a determination unit, processing unit, etc.), etc. Furthermore, the memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0135] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory 402 including instructions, which can be executed by a processor 401 of an electronic device to implement the methods in the above embodiments.

[0136] Optionally, the computer-readable storage medium may be a non-transitory computer-readable storage medium, such as a read-only memory (ROM), a random access memory (RAM), magnetic tape, floppy disk, and optical data storage device.

[0137] In an exemplary embodiment, this application also provides a computer program product including one or more instructions, which can be executed by a processor of an electronic device to perform the methods described above.

[0138] It should be noted that when one or more instructions in the computer-readable storage medium or computer program product are executed by the processor of an electronic device, they implement the various processes of the above method embodiments and achieve the same technical effect as the above method. To avoid repetition, they will not be described again here.

[0139] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0140] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0141] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0142] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0143] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0144] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A road boundary line detection method characterized by, The method comprises: determining whether a determined road boundary line currently exists, and if so, taking the determined road boundary line as a reference boundary line; if not, selecting a radar detection boundary line as the reference boundary line in a case where a lateral distance limit condition is not met, and selecting a forward-looking detection boundary line as the reference boundary line in a case where the lateral distance limit condition is met, the lateral distance limit condition representing that a lateral distance of the forward-looking detection boundary line is less than a lateral distance of the radar detection boundary line; screening candidate fitting points from radar detection points based on the reference boundary line, the candidate fitting points having a lateral distance from the reference boundary line that does not exceed a distance threshold; performing curve fitting based on the candidate fitting points to obtain a boundary fitting line; determining a first fusion weight corresponding to the reference boundary line and a second fusion weight corresponding to the boundary fitting line based on a curvature of the reference boundary line or the boundary fitting line; fusing the reference boundary line and the boundary fitting line based on the first fusion weight and the second fusion weight to obtain a road boundary line. The distance threshold is determined by adding a correction value to a preset lateral distance, and the correction value is determined based on a lateral distance between a candidate fitting point selected at a historical time and the reference boundary line.

2. The road boundary line detection method according to claim 1, characterized by, The radar detection points comprise radar detection base points, and the method further comprises: based on a direction trend of the extension of the forward-looking detection boundary line, performing interval sampling with the radar detection base points as a reference to obtain interval sampling points; adding the interval sampling points to the radar detection points.

3. The road boundary line detection method according to claim 1, characterized by, The method further comprises: determining the first fusion weight and the second fusion weight based on a size relationship between the curvature and a preset curvature; wherein, when the curvature is greater than the preset curvature, the first fusion weight is less than the second fusion weight; and when the curvature is less than the preset curvature, the first fusion weight is greater than the second fusion weight.

4. The road boundary line detection method according to claim 1, characterized by, The method further comprises: performing abnormality checking on the road boundary line based on a number of inner points and / or outer points in the radar detection points; or, performing abnormality checking on the road boundary line based on a cumulative number of stable frames of the road boundary line; or, performing abnormality checking on the road boundary line based on a difference between parameters of a visual lane line and parameters of the road boundary line, wherein the parameters comprise at least one of a parallel state, a slope, and a curvature.

5. A road boundary line detecting apparatus characterized by comprising: The device comprises: a determination module configured to determine whether a determined road boundary line currently exists, and if so, take the determined road boundary line as a reference boundary line; if not, select a radar detection boundary line as the reference boundary line in a case where a lateral distance limit condition is not met, and select a forward-looking detection boundary line as the reference boundary line in a case where the lateral distance limit condition is met, the lateral distance limit condition representing that a lateral distance of the forward-looking detection boundary line is less than a lateral distance of the radar detection boundary line; a screening module configured to screen candidate fitting points from radar detection points based on the reference boundary line, the candidate fitting points having a lateral distance from the reference boundary line that does not exceed a preset distance threshold; a fitting module, configured to perform curve fitting based on the candidate fitting points to obtain a boundary fitting line; a fusion module, specifically configured to: determine a first fusion weight corresponding to the reference boundary line and a second fusion weight corresponding to the boundary fitting line based on curvatures of the reference boundary line and the boundary fitting line; fuse the reference boundary line and the boundary fitting line based on the first fusion weight and the second fusion weight to obtain a road boundary line; the distance threshold is determined based on a preset lateral distance and a correction value; the correction value is determined based on a lateral distance between a candidate fitting point selected at a historical moment and the reference boundary line.

6. A vehicle characterized by comprising: The road boundary line detection device includes the road boundary line detection device of claim 5.

7. An electronic device, comprising: comprises: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the road boundary line detection method of any one of claims 1-4.

Images