Lane line detection method based on multi-source sensors and related device

Abstract

The application relates to the technical field of automatic driving, and discloses a lane line detection method based on a multi-source sensor and related equipment.The method comprises the following steps: a plurality of groups of lane line information in a driving scene are respectively identified by using a multi-source sensor, and the scene type of the driving scene is identified according to the plurality of groups of lane line information, wherein the lane line information comprises a plurality of lane line parameters; according to each lane line parameter in each group of lane line information, the correlation between each sensor is calculated according to a fusion strategy corresponding to the scene type, and target data corresponding to a fusion quantity is screened from each group of lane line information according to the correlation; and according to the target data, lane line parameters of the same type are respectively fused to obtain final lane line information. The application applies two algorithms according to different scenes, expands lane line detection under different driving scenes, takes into account detection speed and accuracy, and flexibly changes the fast detection mode according to different driving scene types.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to a lane line detection method and related equipment based on multi-source sensors. Background Technology

[0002] Multi-source sensor systems based on cameras, LiDAR, and high-precision maps can suffer from reduced detection quality and compromised reliability when using a single sensor for lane line detection during driving. Therefore, existing lane line fusion methods typically combine at least two of these sources—vision-based, LiDAR-based, and high-precision map-based lane lines—to improve detection accuracy.

[0003] However, due to the complex road conditions in driving scenarios, various problems may arise, such as: sudden changes in lighting caused by changes in time (day and night), weather, and road conditions; partial or complete obstruction of lane lines when they are colored, eroded, or blocked; blurred vision caused by inclement weather such as rain and snow; incorrect lane detection due to other similar features such as guardrails, road markings, road surfaces, lane dividers, vehicle lines, tree shadows, etc.; and loss of high-precision map data when entering tunnels or other roads with poor signal. Existing lane line detection methods are unable to take into account the various complex road conditions mentioned above and cannot flexibly switch between rapid detection methods for different types of driving scenarios. Summary of the Invention

[0004] The main objective of this invention is to solve the technical problem that existing lane detection algorithms cannot flexibly change the detection method for different driving scenarios.

[0005] The first aspect of this invention provides a lane line detection method based on multi-source sensors, comprising: identifying multiple sets of lane line information in a driving scenario using multiple source sensors, and identifying the scenario type of the driving scenario based on the multiple sets of lane line information, wherein the lane line information includes multiple lane line parameters; calculating the correlation between each sensor according to a fusion strategy corresponding to the scenario type based on each lane line parameter in each set of lane line information, and selecting a corresponding number of target data from each set of lane line information based on the correlation; and fusing lane line parameters of the same type according to the target data to obtain the final lane line information.

[0006] Optionally, in a first implementation of the first aspect of the present invention, identifying the scenario type of the driving scenario based on the multiple sets of lane line information includes: determining the lane line coordinates corresponding to each sensor based on each lane line parameter in each set of lane line information, and calculating the global similarity distance between every two sets of lane line coordinates; comparing the magnitude of each global similarity distance with a preset first similarity distance threshold, and determining the validity of each set of lane line information based on the comparison results; and determining the scenario type of the driving scenario based on the validity.

[0007] Optionally, in a second implementation of the first aspect of the present invention, the step of calculating the correlation between each sensor according to the fusion strategy corresponding to the scene type based on each lane line parameter in each group of lane line information, and selecting the target data corresponding to the fusion quantity from each group of lane line information based on the correlation includes: if the driving scene is a first scene type, then using each lane line parameter in each group of lane line information under a preset data distribution, constructing the error distance matrix corresponding to each sensor respectively; constructing the relationship matrix corresponding to each sensor according to the error distance matrix corresponding to each lane line parameter and according to a preset error distance threshold; and selecting the target data corresponding to the fusion quantity from each lane line parameter in each group of lane line information based on the value of the relationship matrix corresponding to each lane line parameter.

[0008] Optionally, in a third implementation of the first aspect of the present invention, the step of fusing lane line parameters of the same type according to the target data to obtain the final lane line information includes: fusing lane line parameters of the same type according to the target data and the number of fusions corresponding to each lane line parameter, and according to the data distribution corresponding to each lane line parameter, to obtain the final lane line information, wherein the lane line parameters include the position coordinate information, curvature and slope of the lane line.

[0009] Optionally, in a fourth implementation of the first aspect of the present invention, the step of calculating the correlation between each sensor according to the fusion strategy corresponding to the scene type based on each lane line parameter in each group of lane line information, and filtering the target data corresponding to the fusion quantity from each group of lane line information based on the correlation includes: if the driving scene is a second scene type, then determining the correlation between each sensor based on the validity; filtering the valid lane line information from each group of lane line information to obtain the target data corresponding to the fusion quantity.

[0010] Optionally, in a fifth implementation of the first aspect of the present invention, the step of fusing lane line parameters of the same type according to the target data to obtain the final lane line information includes: determining the fusion weighting coefficient corresponding to each sensor according to the global similarity distance; and fusing lane line parameters of the same type in the target data according to their respective sensors according to the fusion weighting coefficient to obtain the final lane line information.

[0011] Optionally, in a sixth implementation of the first aspect of the present invention, the multi-source sensor includes at least a camera, a radar sensor, and a positioning sensor. The step of using the multi-source sensor to identify multiple sets of lane line information in a driving scenario includes: collecting multiple sets of raw sensor data in the driving scenario using the multi-source sensor, and acquiring historical lane line information for each set of data at a preset frame number from the multi-source sensor; extracting lane line features from the raw sensor data corresponding to the camera, and using the historical lane line information corresponding to the camera to complete the lane line features; identifying the first set of lane line information in the driving scenario according to the completed lane line features; filtering the raw sensor data corresponding to the radar sensor according to preset radar reflection conditions, constructing a point cloud image according to the filtering result, fitting and matching the point cloud image according to the historical lane line information corresponding to the radar sensor to obtain the second set of lane line information in the driving scenario; and extracting high-precision map lane line information from the driving scenario based on the raw sensor corresponding to the positioning sensor, and using this as the third set of lane line information.

[0012] A second aspect of the present invention provides a lane line detection device based on multi-source sensors, comprising: an identification module, configured to identify multiple sets of lane line information in a driving scenario using multiple source sensors, and to identify the scenario type of the driving scenario based on the multiple sets of lane line information, wherein the lane line information includes multiple lane line parameters; a filtering module, configured to calculate the correlation between each sensor according to each lane line parameter in each set of lane line information and a fusion strategy corresponding to the scenario type, and to filter target data of a corresponding fusion quantity from each set of lane line information based on the correlation; and a fusion module, configured to fuse lane line parameters of the same type according to the target data to obtain the final lane line information.

[0013] Optionally, in a first implementation of the second aspect of the present invention, the identification module includes: a similarity distance calculation unit, configured to determine the lane line coordinates corresponding to each sensor based on each lane line parameter in each group of lane line information, and calculate the global similarity distance between every two groups of lane line coordinates; and a scene determination unit, configured to compare the magnitude of each global similarity distance with a preset first similarity distance threshold, and determine the validity of each group of lane line information based on the comparison result; and determine the scene type of the driving scene based on the validity.

[0014] Optionally, in a second implementation of the second aspect of the present invention, the filtering module includes a local filtering unit, configured to: if the driving scenario is a first scenario type, construct error distance matrices between each sensor using each lane line parameter in each group of lane line information under a preset data distribution; construct a relationship matrix between each sensor according to the error distance matrix corresponding to each lane line parameter and a preset error distance threshold; and filter target data of a corresponding fusion quantity from each lane line parameter in each group of lane line information according to the value of the relationship matrix corresponding to each lane line parameter.

[0015] Optionally, in a third implementation of the second aspect of the present invention, the fusion module further includes a local fusion unit, used to: according to the target data and the fusion quantity corresponding to each lane line parameter, fuse lane line parameters of the same type according to the data distribution corresponding to each lane line parameter to obtain the final lane line information, wherein the lane line parameters include the position coordinate information, curvature and slope of the lane line.

[0016] Optionally, in a fourth implementation of the second aspect of the present invention, the filtering module further includes a global filtering unit, used for: if the driving scenario is a second scenario type, determining the correlation between each sensor based on the validity; filtering from each group of lane line information to obtain the target data of the corresponding fusion quantity from the valid lane line information.

[0017] Optionally, in a fifth implementation of the second aspect of the present invention, the fusion module further includes a global fusion unit, configured to: determine the fusion weighting coefficient corresponding to each sensor based on the global similarity distance; and fuse lane line parameters of the same type in the target data according to their respective sensors based on the fusion weighting coefficient to obtain the final lane line information.

[0018] Optionally, in a sixth implementation of the second aspect of the present invention, the multi-source sensor includes at least a camera, a radar sensor, and a positioning sensor, and the identification module further includes: a data acquisition unit, used to acquire multiple sets of raw sensor data in a driving scenario using the multi-source sensor, and to acquire historical lane line information of each set of preset frame numbers from the multi-source sensor; a camera identification unit, used to extract lane line features from the raw sensor data corresponding to the camera, and to complete the lane line features using the historical lane line information corresponding to the camera, and to identify the first set of lane line information in the driving scenario according to the completed lane line features; a radar identification unit, used to filter the raw sensor data corresponding to the radar sensor according to preset radar reflection conditions, and to construct a point cloud image according to the filtering result, and to fit and match the point cloud image according to the historical lane line information corresponding to the radar sensor to obtain the second set of lane line information in the driving scenario; and a map identification unit, used to extract high-precision map lane line information in the driving scenario based on the raw sensor corresponding to the positioning sensor, and to use it as the third set of lane line information.

[0019] A third aspect of the present invention provides a lane detection device based on a multi-source sensor, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor invokes the instructions in the memory to cause the lane detection device based on the multi-source sensor to perform the aforementioned lane detection method based on the multi-source sensor.

[0020] A fourth aspect of the present invention provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the above-described lane detection method based on multi-source sensors.

[0021] The technical solution provided by this invention applies two algorithms according to different scenarios, expanding lane detection across various driving scenarios while balancing detection speed and accuracy. It globally detects whether at least one sensor is unavailable, then determines whether to use a global weighted fusion method or a local Bayesian fusion method. This allows for flexible switching of the detection method for different driving scenario types. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of an embodiment of the lane line detection method based on multi-source sensors in this invention. Figure 2 This is a schematic diagram of another embodiment of the lane line detection method based on multi-source sensors in this invention; Figure 3 This is a schematic diagram of one embodiment of the lane line detection device based on multi-source sensors in this invention. Figure 4 This is a schematic diagram of another embodiment of the lane line detection device based on multi-source sensors in this invention; Figure 5 This is a schematic diagram of one embodiment of the lane line detection device based on multi-source sensors in this invention. Detailed Implementation

[0023] This invention provides a lane line detection method and related equipment based on multi-source sensors. It utilizes multiple sources of sensors to identify multiple sets of lane line information in a driving scenario, and identifies the scenario type based on these multiple sets of lane line information. The lane line information includes multiple lane line parameters. Based on each lane line parameter in each set of lane line information, the correlation between the sensors is calculated according to a fusion strategy corresponding to the scenario type. Based on the correlation, a corresponding number of target data points are selected from each set of lane line information for fusion. Based on the target data, lane line parameters of the same type are fused to obtain the final lane line information. This invention applies two algorithms to different scenarios, expanding lane line detection under different driving scenarios, balancing detection speed and accuracy, and flexibly changing the detection method for different driving scenario types.

[0024] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0025] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 The first embodiment of the lane line detection method based on multi-source sensors in this invention includes: 101. Using multi-source sensors to identify multiple sets of lane line information in a driving scenario, and identifying the scenario type of the driving scenario based on the multiple sets of lane line information, wherein the lane line information includes multiple lane line parameters; It is understood that the executing entity of this invention can be a lane line detection device based on multi-source sensors, or it can be a terminal or a server; no specific limitation is made here. This embodiment of the invention will be described using a server as an example.

[0026] In this embodiment, multi-source sensors refer to sensors used to perceive the environment during vehicle operation, including visual sensors, radar ranging sensors, and positioning sensors. Visual sensors may include monocular stereo vision, binocular stereo vision, panoramic vision, and infrared cameras, while radar ranging sensors include lidar, millimeter-wave radar, and ultrasonic radar. Here, the multi-source sensors include at least a combination of at least two of the different detection types, such as visual sensors, radar ranging sensors, and positioning sensors, to fuse the advantages of sensors with different detection capabilities to identify lane line information. The multi-source sensors include at least a positioning sensor, such as a GPS sensor.

[0027] The vehicle may have one or more sensors of each detection type. For example, for vision sensors, one or more cameras may be installed at the front of the vehicle, positioned parallel to the road surface or at a preset overhead angle, depending on the viewing angle. Alternatively, one or more cameras may be installed at either end of a preset segment of the vehicle, such as the front, middle, or rear of the vehicle (divided into three equal parts or other preset proportions). The final single image or combination of multiple images from the one or more cameras must include at least the lane lines of the current lane in the direction of the vehicle's travel, and may also include the front portion of the vehicle, adjacent lane lines, and / or other vehicles in the driving scenario.

[0028] Specifically, in the multi-source sensor system, each group of sensors of the same detection type identifies a set of lane line information in the driving scenario. For radar-type ranging sensors, the driver assistance controller receives the transmitted signal from a LiDAR (Light Detection and Ranging) sensor, filters out lane line point clouds based on the LiDAR's reflection wavelength and intensity, and then fits the lane line point clouds to obtain the final lane line information. For example, the echo intensity of the lane line to the LiDAR transmitted signal is 12Hz-30Hz, the echo intensity of the asphalt and concrete road surface to the LiDAR transmitted signal is 5Hz-8Hz, and the echo intensity of the vehicle, street light, road sign, and other metal objects to the LiDAR transmitted signal is 45Hz-150Hz. The received signal is filtered according to the echo intensity of 12Hz-30Hz, and the lane line point cloud is generated based on the processing result. For vision-type sensors, the driver assistance controller receives image information from the camera, uses a deep learning model to extract lane line feature information from the image information, extracts lane line information based on the feature information, and finally performs lane line fitting to obtain the lane line information.

[0029] Lane line information can include lane line parameters such as three-dimensional spatial information, curvature information, and / or slope information. High-precision map data is obtained based on lane line information from at least positioning sensors. The type of driving scenario is then identified based on this high-precision map data, including scenarios with no or weak signal traffic where high-precision map data is easily lost, and scenarios with normal signal traffic.

[0030] 102. Based on the lane line parameters in each group of lane line information, calculate the correlation between each sensor according to the fusion strategy corresponding to the scene type, and based on the correlation, select the target data corresponding to the fusion quantity from each group of lane line information. In this embodiment, under different scenario types, corresponding fusion strategies are preset to calculate the correlation between sensors of different detection types and to perform target data filtering. The setting criteria for the fusion strategy include at least the scenario type of the driving scenario, and the corresponding settings include calculating the correlation and filtering the target data for each lane line parameter independently, and / or calculating the correlation and filtering the target data for all lane line parameters as a whole.

[0031] Among them, each lane line parameter in each group of lane line information from multi-source sensors, such as the lane line parameters obtained by lidar detection ( , , , Lane line parameters obtained by camera recognition ( , ), , , , Lane line parameters of high-precision maps obtained from GPS sensor positioning ( , , , , , represent the X coordinate, Y coordinate, C curvature, and S slope of the lane line, respectively. Based on the lane line parameters in each group of lane line information, the correlation between each pair of sensors is calculated according to the detection type. Based on the correlation, the correct lane line information and the lane line information with anomalies are determined. Based on the determined results, the fusion quantity of the corresponding lane line information is obtained, and the corresponding lane line information is selected as the target data.

[0032] For example, in a scenario with no or weak traffic signal, the lane line information output by the positioning sensor can be determined to be unreliable and can be directly eliminated, reducing the computational data consumption for correlation calculation and target data screening. In a scenario with normal traffic signal, the lane line information output by the positioning sensor is determined to be reliable. This information is used as a screening criterion to calculate the correlation between the lane line information and other detection types of sensors, thereby determining the reliability of the lane line information detected by other detection types of sensors and screening the target data. 103. Based on the target data, merge lane line parameters of the same type to obtain the final lane line information.

[0033] In this embodiment, after the target data is obtained through screening, lane line parameters of the same location coordinates and type are fused based on the location coordinates of the target data obtained by each type of sensor. Among them, the corresponding lane line parameters are fused based on the fusion strategy selected for different scene types.

[0034] For example, in driving scenarios with no or weak traffic signals, all lane line parameters are treated as a whole, and the lane line parameters at each coordinate in the lane line information are weighted according to a preset correlation to obtain the final lane line information. In driving scenarios with normal traffic signals, each individual lane line parameter of the same type in the target data is fused according to its coordinate position to obtain the final lane line information.

[0035] In this embodiment of the invention, two algorithms are applied according to different scenarios to expand lane detection under various driving scenarios, balancing detection speed and accuracy. The system globally detects whether at least one sensor is unavailable, and then decides whether to use a global weighted fusion method or a local Bayesian fusion method. This allows for flexible switching of the detection method for different driving scenario types.

[0036] Please see Figure 2 Another embodiment of the lane line detection method based on multi-source sensors in this invention includes: 201. Use multi-source sensors to identify multiple sets of lane line information in the driving scenario, and determine the lane line coordinates corresponding to each sensor based on each lane line parameter in each set of lane line information, and calculate the global similarity distance between each pair of lane line coordinates. In this embodiment, the multi-source sensor includes at least a camera, a radar sensor, and a positioning sensor. The step of using the multi-source sensor to identify multiple sets of lane line information in a driving scenario includes: collecting multiple sets of raw sensor data in the driving scenario using the multi-source sensor, and obtaining historical lane line information for each set of data at a preset frame rate from the multi-source sensor; extracting lane line features from the raw sensor data corresponding to the camera, and using the historical lane line information corresponding to the camera to complete the lane line features; identifying the first set of lane line information in the driving scenario based on the completed lane line features; filtering the raw sensor data corresponding to the radar sensor according to preset radar reflection conditions, constructing a point cloud image based on the filtering result, fitting and matching the point cloud image according to the historical lane line information corresponding to the radar sensor to obtain the second set of lane line information in the driving scenario; and extracting high-precision map lane line information from the driving scenario based on the raw sensor data corresponding to the positioning sensor, and using this as the third set of lane line information.

[0037] In this embodiment, the lane line parameters in each group of lane line information are treated as a whole. For example, it is determined whether at least one group of sensors in each of the three detection types is abnormal. The lane line coordinates can be directly obtained through the lane line parameters. The global similarity distance between each pair of lane line coordinates is calculated. If neither pair of sensors is abnormal, the two pairs of lane line coordinates should be the same, or the similarity between them should be high (preset similarity) due to errors caused by environmental or other factors. Otherwise, at least one pair of sensors is abnormal.

[0038] 202. Compare the magnitudes of each global similarity distance with the preset first similarity distance threshold, and determine the validity of each group of lane line information based on the comparison results; 203. Based on the validity, determine the scenario type of the driving scenario, wherein the lane line information includes multiple lane line parameters; In this embodiment, based on each global similarity distance (i.e., the aforementioned similarity), it is compared with a preset first similarity distance threshold. If the similarity is lower than the first similarity distance threshold, it is determined that the similarity between the two sets of lane line information is low, and at least one set of lane line information is invalid. If the similarity is higher than the first similarity distance threshold, it is determined that the similarity between the two sets of lane line information is high, and both sets of lane line information are considered valid.

[0039] In this embodiment, if at least one set of lane line information is invalid, the driving scenario is determined to be a no-signal / weak-signal driving scenario. Alternatively, it can be a driving scenario where visual sensors experience reduced detection accuracy due to factors such as weather, time, or lane line coloring. Or it can be a driving scenario where distance sensors experience reduced detection accuracy due to abnormal signal feedback, such as lane line obstruction, guardrails, or road markings (second scenario type). If all sets of lane line information are valid, the driving scenario is determined to be a normal driving scenario (first scenario type).

[0040] Specifically, for example, the lane line parameters of LiDAR measurement data are (ai1, ai2, ai3, ai4), the lane line parameters of camera detection data are (bi1, bi2, bi3, bi4), and the lane line parameters of high-precision map data are (ci1, ci2, ci3, ci4), representing the X-coordinate, Y-coordinate, curvature, and slope of the lane line at the i-th point, respectively. I is a positive integer. For example, the squared difference between any two sets of lane line parameters is calculated as follows:

[0041] For example, if A is less than the preset difference, it means that both the lidar measurement data and the camera detection data are valid; if A is greater than the preset difference, it means that at least one of the lidar measurement data and the camera detection data is invalid, and so on.

[0042] 204. If the driving scenario is the first scenario type, then each lane line parameter in each group of lane line information under the preset data distribution is used to construct the corresponding error distance matrix between each sensor. 205. Based on the error distance matrix corresponding to each lane line parameter, construct the relationship matrix between each sensor according to the preset error distance threshold; 206. Based on the values ​​of the relation matrix corresponding to each lane line parameter, select the target data corresponding to the fusion quantity from each lane line parameter of each group of lane line information; In this embodiment, it is assumed that each set of lane line parameters, such as camera measurement data, LiDAR measurement data, and high-precision map data, follows a normal distribution. Taking the lane line X-coordinate as an example, the lane line X-coordinate is X ~ N (μ 0， σ 2 ), measurement data k = 1, 2, 3. K=1 represents camera measurement data; K=2 represents lidar measurement data; K=3 represents high-precision map data.

[0043] Specifically, the confidence distance of the three sensors is calculated as the error distance: ,in, Then confidence distance (e.g., i, j = 1, 2, 3) will form a confidence distance matrix as the error distance matrix: Then, based on the preset distance threshold... As a preset error distance threshold, based on The corresponding relation matrix is ​​constructed as follows: = Finally, based on the relationship matrix, the target data is filtered to select the optimal number of data to be fused, m, resulting in the target data A = (A1, A2, ..., A...). m ) .

[0044] 207. Based on the target data and the number of fusions corresponding to each lane line parameter, and according to the data distribution corresponding to each lane line parameter, the same type of lane line parameters are fused to obtain the final lane line information. The lane line parameters include the position coordinate information, curvature and slope of the lane line.

[0045] In this embodiment, based on the fusion quantity and data distribution corresponding to the lane line parameters, the X coordinate of the fused lane line is... Similarly, the Bayesian fusion values ​​of the Y-coordinate, curvature, and slope of the lane line can be obtained.

[0046] In this embodiment, if the driving scenario is a second scenario type, then the step of calculating the correlation between each sensor according to the fusion strategy corresponding to the scenario type based on the lane line parameters in each group of lane line information, and filtering the target data corresponding to the fusion quantity from each group of lane line information based on the correlation includes: determining the correlation between each sensor based on the validity; and filtering the valid lane line information from each group of lane line information to obtain the target data corresponding to the fusion quantity.

[0047] Specifically, for the second type of driving scenario, based on the validity of each sensor calculated earlier, if both sets of lane line information are valid, then the two sets of sensors are determined to be related; otherwise, they are not related. One set of lane line information is selected from the related or unrelated lane line information, and the number of selected lane line information sets is counted to obtain the corresponding fusion quantity and target data.

[0048] In one implementation, the step of fusing lane line parameters of the same type according to the target data to obtain the final lane line information includes: determining the fusion weighting coefficient corresponding to each sensor according to the global similarity distance; and fusing lane line parameters of the same type in the target data according to their respective sensors according to the fusion weighting coefficient to obtain the final lane line information.

[0049] Specifically, based on the filtered target data, the fused lane line information di = (Ji (ai1,ai2,ai3,ai4)+Ki(bi1,bi2,bi3,bi4)+Li(ci1,ci2,ci3,ci4)) / (Ji+Ki+Li). Where Ji, ki, and Li are weighting coefficients with a range of (0, 1); the weighting coefficient is 0 when the above squared difference is invalid; the smaller the value of the squared difference, the larger the weighting coefficient.

[0050] The lane line detection method based on multi-source sensors in the embodiments of the present invention has been described above. The lane line detection device based on multi-source sensors in the embodiments of the present invention will be described below. Please refer to [link / reference]. Figure 3 One embodiment of the lane line detection device based on multi-source sensors in this invention includes: The identification module 301 is used to identify multiple sets of lane line information in a driving scenario using multi-source sensors, and to identify the scenario type of the driving scenario based on the multiple sets of lane line information, wherein the lane line information includes multiple lane line parameters. The filtering module 302 is used to calculate the correlation between each sensor according to the fusion strategy corresponding to the scene type based on each lane line parameter in each group of lane line information, and to filter the target data corresponding to the fusion quantity from each group of lane line information based on the correlation. The fusion module 303 is used to fuse lane line parameters of the same type according to the target data to obtain the final lane line information.

[0051] In this embodiment of the invention, two algorithms are applied according to different scenarios to expand lane detection under various driving scenarios, balancing detection speed and accuracy. The system globally detects whether at least one sensor is unavailable, and then decides whether to use a global weighted fusion method or a local Bayesian fusion method. This allows for flexible switching of the detection method for different driving scenario types.

[0052] Please see Figure 4 Another embodiment of the lane line detection device based on multi-source sensors in this invention includes: The identification module 301 is used to identify multiple sets of lane line information in a driving scenario using multi-source sensors, and to identify the scenario type of the driving scenario based on the multiple sets of lane line information, wherein the lane line information includes multiple lane line parameters. The filtering module 302 is used to calculate the correlation between each sensor according to the fusion strategy corresponding to the scene type based on each lane line parameter in each group of lane line information, and to filter the target data corresponding to the fusion quantity from each group of lane line information based on the correlation. The fusion module 303 is used to fuse lane line parameters of the same type according to the target data to obtain the final lane line information.

[0053] Specifically, the identification module 301 includes: The similarity distance calculation unit 3011 is used to determine the lane line coordinates corresponding to each sensor based on the lane line parameters in each group of lane line information, and to calculate the global similarity distance between each pair of lane line coordinates. The scene determination unit 3012 is used to compare the magnitude of each global similarity distance with a preset first similarity distance threshold, and determine the validity of each group of lane line information based on the comparison results; and determine the scene type of the driving scene based on the validity.

[0054] Specifically, the filtering module 302 includes a local filtering unit 3021, used for: If the driving scenario is the first scenario type, then each lane line parameter in each group of lane line information under the preset data distribution is used to construct the error distance matrix between each sensor respectively. Based on the error distance matrix corresponding to each lane line parameter, and according to the preset error distance threshold, construct the corresponding relationship matrix between each sensor; Based on the values ​​of the relation matrix corresponding to each lane line parameter, target data of the corresponding fusion quantity are selected from each lane line parameter of each group of lane line information.

[0055] Specifically, the fusion module 303 includes a local fusion unit 3031, used for: Based on the target data and the number of fusions corresponding to each lane line parameter, lane line parameters of the same type are fused according to the data distribution corresponding to each lane line parameter to obtain the final lane line information. The lane line parameters include the position coordinate information, curvature and slope of the lane line.

[0056] Optionally, in a fourth implementation of the first aspect of the present invention, the filtering module 302 further includes a global filtering unit 3022, used for: If the driving scenario is of the second scenario type, then the correlation between the various sensors is determined based on the validity. From each group of lane line information, the effective lane line information is filtered to obtain the target data of the corresponding fusion quantity.

[0057] Specifically, the fusion module 303 further includes a global fusion unit 3032, used for: The fusion weighting coefficients for each sensor are determined based on the global similarity distance. Based on the fusion weighting coefficients, lane line parameters of the same type in the target data are fused according to their respective sensors to obtain the final lane line information.

[0058] Specifically, the multi-source sensor includes at least a camera, a radar sensor, and a positioning sensor, and the identification module 301 further includes: The acquisition unit 3013 is used to acquire multiple sets of raw sensor data in a driving scenario using multi-source sensors, and to obtain historical lane line information of each set of preset frames from the multi-source sensors. The camera recognition unit 3014 is used to extract lane line features from the original sensor data corresponding to the camera, and use the historical lane line information corresponding to the camera to complete the lane line features, and recognize the first set of lane line information in the driving scenario according to the completed lane line features. The radar identification unit 3015 is used to filter the original sensing data corresponding to the radar sensor according to the preset radar reflection conditions, construct a point cloud image according to the filtering result, and fit and match the point cloud image according to the historical lane line information corresponding to the radar sensor to obtain the second set of lane line information in the driving scenario. The map recognition unit 3016 is used to extract high-precision map lane line information in the driving scenario based on the original sensor corresponding to the positioning sensor, and use it as the third set of lane line information.

[0059] above Figure 3 and Figure 4 The lane line detection device based on multi-source sensors in the embodiments of the present invention will be described in detail from the perspective of modular functional entities. The lane line detection device based on multi-source sensors in the embodiments of the present invention will be described in detail from the perspective of hardware processing.

[0060] Figure 5This is a schematic diagram of a lane line detection device based on a multi-source sensor according to an embodiment of the present invention. The lane line detection device 500 based on the multi-source sensor can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 510 (e.g., one or more processors) and a memory 520, and one or more storage media 530 (e.g., one or more mass storage devices) for storing application programs 533 or data 532. The memory 520 and storage media 530 can be temporary or persistent storage. The program stored in the storage media 530 may include one or more modules (not shown in the diagram), each module may include a series of instruction operations on the lane line detection device 500 based on the multi-source sensor. Furthermore, the processor 510 may be configured to communicate with the storage media 530 and execute the series of instruction operations in the storage media 530 on the lane line detection device 500 based on the multi-source sensor.

[0061] The lane detection device 500 based on multi-source sensors may also include one or more power supplies 540, one or more wired or wireless network interfaces 550, one or more input / output interfaces 560, and / or one or more operating systems 531, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that... Figure 5 The illustrated lane line detection device structure based on multi-source sensors does not constitute a limitation on lane line detection devices based on multi-source sensors. It may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0062] The present invention also provides a lane line detection device based on multi-source sensors. The computer device includes a memory and a processor. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor performs the steps of the lane line detection method based on multi-source sensors in the above embodiments.

[0063] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the steps of the lane line detection method based on multi-source sensors.

[0064] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0065] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0066] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-source sensor based lane line detection method, characterized in that, The lane line detection method based on multi-source sensors includes: Multiple sets of lane line information in a driving scenario are identified using multi-source sensors, and the scenario type of the driving scenario is identified based on the multiple sets of lane line information, wherein the lane line information includes multiple lane line parameters; Based on the lane line parameters in each group of lane line information, the correlation between each sensor is calculated according to the fusion strategy corresponding to the scene type. Based on the correlation, the target data corresponding to the fusion quantity is filtered from each group of lane line information. The fusion strategy is based on the scene type preset, and the corresponding settings include calculating the correlation and filtering the target data for each lane line parameter independently, and / or calculating the correlation and filtering the target data for all lane line parameters as a whole. Based on the target data, lane line parameters of the same type are fused to obtain the final lane line information; The step of calculating the correlation between each sensor according to the fusion strategy corresponding to the scene type based on each lane line parameter in each group of lane line information, and selecting the target data of the corresponding fusion quantity from each group of lane line information based on the correlation includes: if the driving scene is a first scene type, then using each lane line parameter in each group of lane line information under a preset data distribution, constructing an error distance matrix respectively; constructing a relationship matrix according to the error distance matrix and a preset error distance threshold; and selecting the target data of the corresponding fusion quantity from each lane line parameter in each group of lane line information based on the value of the relationship matrix.

2. The lane line detection method based on multi-source sensors according to claim 1, characterized in that, The scenario type identified based on the multiple sets of lane line information includes: Based on the lane line parameters in each group of lane line information, determine the lane line coordinates corresponding to each sensor and calculate the global similarity distance between each pair of lane line coordinates. The magnitudes of each global similarity distance and the preset first similarity distance threshold are compared, and the validity of each group of lane line information is determined based on the comparison results. Based on the validity, the scenario type of the driving scenario is determined.

3. The lane line detection method based on multi-source sensors according to claim 1, characterized in that, The step of fusing lane line parameters of the same type according to the target data to obtain the final lane line information includes: Based on the target data and the number of fusions corresponding to each lane line parameter, lane line parameters of the same type are fused according to the data distribution corresponding to each lane line parameter to obtain the final lane line information. The lane line parameters include the position coordinate information, curvature and slope of the lane line.

4. The lane line detection method based on multi-source sensors according to claim 2, characterized in that, The step of calculating the correlation between each sensor according to the fusion strategy corresponding to the scene type based on the lane line parameters in each group of lane line information, and then selecting the target data of the corresponding fusion quantity from each group of lane line information based on the correlation includes: If the driving scenario is of the second scenario type, then the correlation between the various sensors is determined based on the validity. Valid lane line information is filtered from each group of lane line information to obtain the target data of the corresponding fusion quantity.

5. The lane line detection method based on multi-source sensors according to claim 4, characterized in that, The step of fusing lane line parameters of the same type according to the target data to obtain the final lane line information includes: The fusion weighting coefficients for each sensor are determined based on the global similarity distance. Based on the fusion weighting coefficients, lane line parameters of the same type in the target data are fused according to their respective sensors to obtain the final lane line information.

6. The lane line detection method based on multi-source sensors according to claim 1, characterized in that, The multi-source sensor includes at least a camera, a radar sensor, and a positioning sensor, and the step of using the multi-source sensor to identify multiple sets of lane line information in a driving scenario includes: Multiple sets of raw sensor data in a driving scenario are collected using multi-source sensors, and historical lane line information of each set of preset frames from the multi-source sensors is obtained. Extract lane line features from the original sensor data corresponding to the camera, and use the historical lane line information corresponding to the camera to complete the lane line features. Then, identify the first set of lane line information in the driving scenario according to the completed lane line features. The original sensing data corresponding to the radar sensor is filtered according to the preset radar reflection conditions, and a point cloud image is constructed according to the filtering result. The point cloud image is fitted and matched according to the historical lane line information corresponding to the radar sensor to obtain the second set of lane line information in the driving scenario. Based on the original sensing data corresponding to the positioning sensor, high-precision map lane line information in the driving scenario is extracted and used as the third set of lane line information.

7. A lane line detection device based on multi-source sensors, characterized in that, The lane detection device based on multi-source sensors includes: The identification module is used to identify multiple sets of lane line information in a driving scenario using multi-source sensors, and to identify the scenario type of the driving scenario based on the multiple sets of lane line information, wherein the lane line information includes multiple lane line parameters; The filtering module is used to calculate the correlation between each sensor according to the fusion strategy corresponding to the scene type based on each lane line parameter in each group of lane line information, and to filter the target data corresponding to the fusion quantity from each group of lane line information based on the correlation. The fusion strategy is based on the scene type preset, and the corresponding settings include calculating the correlation and filtering the target data for each lane line parameter independently, and / or calculating the correlation and filtering the target data for all lane line parameters as a whole. The fusion module is used to fuse lane line parameters of the same type according to the target data to obtain the final lane line information. The filtering module includes a local filtering unit, used for: if the driving scenario is a first scenario type, constructing an error distance matrix by using each lane line parameter in each group of lane line information under a preset data distribution; constructing a relationship matrix according to the error distance matrix and a preset error distance threshold; and filtering the corresponding number of target data from each lane line parameter in each group of lane line information according to the value of the relationship matrix.

8. A lane line detection device based on multi-source sensors, characterized in that, The lane detection device based on multi-source sensors includes: a memory and at least one processor, wherein the memory stores instructions; The at least one processor invokes the instructions in the memory to cause the multi-source sensor-based lane detection device to perform the steps of the multi-source sensor-based lane detection method as described in any one of claims 1-6.

9. A computer-readable storage medium storing instructions thereon, characterized in that, When the instructions are executed by the processor, they implement the steps of the lane line detection method based on multi-source sensors as described in any one of claims 1-6.

Images