Lane line positioning method and device, electronic device and storage medium

Abstract

The present application discloses a lane line positioning method and device, an electronic device, and a storage medium, wherein the method comprises extracting lane line information of the current road from high-precision map data based on positioning information provided by a vehicle integrated navigation module and the current driving direction of the vehicle; matching the lane line information of the current road with a segmentation result obtained by panoramic segmentation of an image of the current road, the image of the current road being obtained based on a vehicle visual perception module; determining the lane in which the vehicle is located based on the matching result of the panoramic segmentation of the lane line information of the current road and the lane line information in the segmentation result obtained by panoramic segmentation of the image of the current road; and performing lane line point cloud matching based on the lane in which the vehicle is located to achieve lateral positioning of the vehicle position and correction of the heading angle. This application reduces annotation costs and improves the accuracy of lane line matching.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to a lane positioning method and device, an electronic device, and a storage medium. Background Art

[0002] Integrated navigation, which uses IMU + GPS / RTK, achieves high-precision positioning. However, since integrated navigation is affected by the quality of GNSS signals, interference with GNSS signals can lead to significant errors, causing the autonomous vehicle's positioning to deviate from its lane or even the road.

[0003] Lateral positioning correction technology using lane lines is widely used in multi-sensor fusion positioning. By using the lane lines extracted by the perception module and matching them with the lane lines in the high-precision map, the autonomous vehicle can be guaranteed not to deviate from the lane. The lane line matching algorithms in related technologies mainly include:

[0004] Method 1 uses a point cloud matching algorithm to match a point cloud generated from lane line pixels obtained through image segmentation with a virtual point cloud generated from lane lines in the HD map. However, this can result in mismatches when there are many lane lines on the road and / or when lane lines in the image are not fully recognized.

[0005] Method 2 uses a lane line equation (lane line type + lane line color) generated by a lane line recognition algorithm and matches it with the lane line equation generated by the HD map. However, this method requires high annotation costs and high model accuracy, and the results may be inaccurate due to factors such as vehicle occlusion and lighting. Summary of the Invention

[0006] The embodiments of the present application provide a lane line positioning method and related devices to reduce the labeling cost and improve the accuracy of lane line matching.

[0007] The embodiments of this application adopt the following technical solutions:

[0008] In a first aspect, an embodiment of the present application provides a lane line positioning method, wherein the method includes:

[0009] Extracting lane line information of the current road from the high-precision map data based on the positioning information provided by the vehicle integrated navigation module and the current driving direction of the vehicle, wherein the lane line information includes at least lane line color, lane line type, and lane topology relationship;

[0010] matching lane line information of the current traveling road with a segmentation result obtained by performing panoptic segmentation on an image of the current traveling road, the image of the current traveling road being obtained according to a vehicle visual perception module;

[0011] determining a lane in which the vehicle is located based on a matching result of the lane line information of the current traveling road and the lane line information in the segmentation result obtained by performing panoptic segmentation on the image of the current traveling road;

[0012] Lane line point cloud matching is performed according to the lane where the vehicle is located to achieve lateral positioning of the vehicle position and correction of the heading angle.

[0013] In some embodiments, the determination of the vehicle's lane may be based on at least one of a plurality of matching strategies, including: a matching strategy based on lane color, a matching strategy based on lane topology, and a matching strategy based on lane change determination. The determination of the vehicle's lane includes:

[0014] Determining the current lane of the vehicle based on the confidence levels of the output results of the multiple matching strategies;

[0015] Alternatively, when two or more output results of the multiple matching strategies are the same, the lane in which the vehicle is currently located is determined.

[0016] In some embodiments, determining the lane of the vehicle based on the lane line information of the current road and the matching result of the panoptic segmentation of the lane line information in the segmentation result obtained after performing panoptic segmentation on the image of the current road includes:

[0017] Setting a color category of the lane line color in the lane line information of the current driving road according to a matching result of the lane line color in the lane line information;

[0018] Clustering original pixel points in the lane line obtained by panoptic segmentation according to the number of color categories of the lane line color;

[0019] According to the clustering results, the lane lines segmented by the panorama are assigned colors;

[0020] The lane lines are matched according to the color assignment results, and the color of the lane where the vehicle is currently located is calculated.

[0021] In some embodiments, determining the lane of the vehicle based on a matching result between lane line information of the current road and lane line information in a segmentation result obtained by performing panoptic segmentation on the image of the current road includes:

[0022] Calculating the rightmost lane based on the lane line topology extracted from the high-precision map data and the drivable area, road shoulder, and lane lines in the segmentation result obtained after the panoptic segmentation;

[0023] If the drivable area in the image contains the drivable area of ​​the entire road, the distance from the vehicle to the edge of the rightmost lane is calculated;

[0024] The lane in which the vehicle is currently located is determined based on the distance between the vehicle and the edge of the rightmost lane.

[0025] In some embodiments, determining the panoptic segmentation of the lane in which the vehicle is located based on a matching result between the lane line information of the current road and the lane line information in the segmentation result obtained by performing panoptic segmentation on the image of the current road further includes:

[0026] Determining the lane information of the vehicle at the previous moment and the vehicle's heading angle change rate information based on the matching result of the lane line information, and judging whether the vehicle is currently performing a lane change operation;

[0027] The lane of the vehicle at the current moment is matched according to the lane change operation judgment result to determine the current lane of the vehicle.

[0028] In some embodiments, the method further includes: if the vehicle is equipped with a dynamic object detection module, using the detection results of the dynamic object detection module to process the lane lines in the segmentation results obtained after panoptic segmentation to determine whether the lane line type is a dashed line; if the lane line type is a dashed line, matching the dashed lane line type in the lane line information of the current road with the segmentation result obtained after panoptic segmentation of the image of the current road. In some embodiments, before extracting all lane line information of the current road from the high-precision map data based on the positioning information provided by the vehicle integrated navigation module and the current driving direction of the vehicle, the method further includes:

[0029] When the RTK in the vehicle integrated navigation module is a fixed solution, calculating a lateral correction error parameter of the current lane line matching algorithm, and when the RTK is a fixed solution, the positioning state of the vehicle integrated navigation module is an available state;

[0030] When the current positioning status of the vehicle's integrated navigation module is available and within a preset track deduction time threshold, lateral positioning correction of the vehicle position based on lane line matching is started according to the lateral correction error parameter.

[0031] In a second aspect, an embodiment of the present application further provides a lane line positioning device, wherein the device includes:

[0032] An extraction module is used to extract lane line information of the current road from the high-precision map data based on the positioning information provided by the vehicle integrated navigation module and the current driving direction of the vehicle. The lane line information includes at least lane line color, lane line type, and lane topology relationship;

[0033] a matching module, configured to match lane line information of the current traveling road with a segmentation result obtained by performing panoptic segmentation on an image of the current traveling road, the image of the current traveling road being obtained according to the vehicle visual perception module;

[0034] a lane determination module, configured to determine a lane in which a vehicle is located based on a matching result of the lane line information of the current traveling road and the lane line information in the segmentation result obtained by performing panoptic segmentation on the image of the current traveling road;

[0035] The positioning correction module is used to perform lane line point cloud matching based on the lane where the vehicle is located, so as to realize the lateral positioning of the vehicle position and the correction of the heading angle.

[0036] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor; and a memory arranged to store computer-executable instructions, wherein the executable instructions, when executed, enable the processor to perform the above method.

[0037] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores one or more programs. When the one or more programs are executed by an electronic device including multiple application programs, the electronic device executes the above method.

[0038] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects:

[0039] All lane information for the current road is extracted from high-precision map data as a priori information, and then matched to the lane information obtained after panoramic segmentation of the current road. This avoids the information deficiency of pure segmentation models or incorrect color / category classification by recognition models. Furthermore, the segmentation results obtained after panoramic segmentation and the matching results of lane information can reduce the manual annotation costs required for lane identification and classification. By combining multiple matching strategies to calculate the vehicle's lane, the uncertainty associated with a single strategy is eliminated. Finally, lane point cloud matching is performed based on the vehicle's lane to achieve lateral positioning and heading angle correction of the vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0041] Figure 1 This is a flow chart of the lane line positioning method in an embodiment of the present application;

[0042] Figure 2This is a schematic structural diagram of a lane line positioning device in an embodiment of the present application;

[0043] Figure 3 This is a schematic structural diagram of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0044] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this application and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0045] The following describes in detail the technical solutions provided by various embodiments of the present application in conjunction with the accompanying drawings.

[0046] The present application provides a lane positioning method for autonomous driving. Figure 1 As shown, a flow chart of a lane line positioning method for autonomous driving according to an embodiment of the present application is provided. The method includes at least the following steps S210 to S240:

[0047] Step S210: Extract lane line information of the current road from the high-precision map data based on the positioning information provided by the vehicle integrated navigation module and the current driving direction of the vehicle. The lane line information includes at least lane line color, lane line type, and lane topology relationship.

[0048] It's understandable that using a lane-line-based lateral positioning correction method (algorithm) can ensure that autonomous vehicles stay within their lanes. However, this method (algorithm) is only effective if the positioning information obtained by the vehicle's integrated navigation module, IMU+GPS / RTK, is usable or the error during track derivation can be adjusted.

[0049] In one embodiment of the present application, before the lateral positioning correction of the vehicle position, it also includes: when the RTK in the vehicle's integrated navigation module is a fixed solution, calculating the lateral correction error parameter of the current lane line matching algorithm, and when the RTK state is a fixed solution, the positioning state of the vehicle's integrated navigation module is an available state; when the positioning state of the vehicle's current integrated navigation module is an available state and is within a preset track deduction time threshold range, starting the lateral positioning correction of the vehicle position based on lane line matching according to the lateral correction error parameter.

[0050] In specific implementations, when the RTK state is determined to be a fixed solution based on a good GNSS signal, the vehicle's integrated navigation module's positioning information is available. At this point, the lateral correction error of the current lane matching algorithm is calculated and updated based on the actual situation. This is to say, the lateral correction error err is introduced to adapt to the camera's internal and external parameters.

[0051] Furthermore, based on the statistical results of actual tests (historical data), the time threshold Thre_dr of track deduction is calculated. That is, the error of the track deduction of the IMU in the vehicle integrated navigation module can be corrected by the lane matching algorithm within a certain period of time, and lane matching is performed based on this error range.

[0052] It is understood that if the error is exceeded, it is considered uncorrectable and the positioning information is outdated.

[0053] It's important to note that when using track derivation, accumulated errors can cause the vehicle's trajectory to deviate significantly and erratically after exceeding a threshold, which cannot be corrected. Furthermore, with RTK using single-point or floating-point solutions, while the vehicle may experience meter-level deviations, this can be corrected if the autonomous vehicle remains on track or deviates slightly from the road.

[0054] Finally, it is necessary to start the lane matching algorithm to calculate lateral correction based on the current positioning status of the vehicle integrated navigation module (within the preset error range or positioning accuracy is accurate to the centimeter / millimeter level) or according to actual needs. It can be understood that when the positioning status of the vehicle integrated navigation module is available, that is, the RTK status is a fixed solution, no lateral correction is required. When the positioning status of the vehicle integrated navigation module is unavailable, lateral correction is required.

[0055] At this point, the lane matching algorithm has been successfully initialized and the lateral correction error err has been calculated. The lateral correction error err will no longer be updated subsequently, and the lateral correction function needs to be used continuously.

[0056] After the aforementioned pre-judgment, all lane line information of the current driving road can be extracted from the high-precision map data based on the positioning information in the vehicle integrated navigation module and the current driving direction of the vehicle.

[0057] In this case, the HD map data is pre-collected and generated within the preset driving area. This means that the integrated navigation module receives the positioning information received by the current integrated navigation module and, based on this positioning information and the vehicle's driving direction, extracts all lane marking information for the current lane in the HD map data.

[0058] For example, in order to prevent the vehicle from deviating into the opposite lane in combination with the vehicle's driving direction, all lane line information corresponding to the current driving road based on positioning information (centimeter / millimeter level positioning accuracy) is extracted from the high-precision map data. This lane line information includes the type and color of the lane line. The types of lane lines mainly include but are not limited to solid lines and dashed lines. The colors of lane lines mainly include but are not limited to white and yellow. In normal scenarios, solid lines, dashed lines, white lines, and yellow lines will all appear.

[0059] Step S220 , matching the lane line information of the current driving road with a segmentation result obtained by performing panoramic segmentation on an image of the current driving road, where the image of the current driving road is obtained according to a vehicle visual perception module.

[0060] Using all lane line information of the current road as a priori conditions, the lane line information of the current road obtained after panoptic segmentation is matched. In other words, for each frame of the current road image, the panoptic segmentation results are used to match the lane line information. As mentioned above, the lane line information here may include different lane line types and colors, such as solid, dashed, white, and yellow.

[0061] By using the lane line data of the high-precision map as a priori conditions after panoramic segmentation, the type and color of the lane lines are calculated and matched, avoiding the lack of information in the pure semantic segmentation model or the incorrect color classification of the semantic recognition model.

[0062] Exemplarily, a panoramic segmentation result is obtained for each frame of the image, and the obtained segmentation result is combined with all lane line information of the current driving road as a priori conditions, and the lane line information in the segmentation result is matched and calculated again to obtain the corresponding lane line color and color number, lane line type and type number.

[0063] It should be noted that the segmentation result obtained after panoramic segmentation includes the semantic segmentation results of all elements in the current frame image, including the segmentation results for lane lines.

[0064] Step S230 , determining the lane in which the vehicle is located based on the lane line information of the current driving road and the matching result of the lane line information in the segmentation result obtained by performing panoramic segmentation on the image of the current driving road.

[0065] Based on the lane information matching results, the vehicle's lane is calculated and determined based on the segmentation results obtained after panoptic segmentation. In other words, the vehicle's current lane can be determined by combining the different segmentation results obtained after panoptic segmentation with the lane information matching results. By using high-precision map data and the segmentation results obtained after panoptic segmentation (driving area, road shoulder) as prior information, the manual annotation cost of lane identification / classification is greatly reduced. The lane identification process utilizes the aforementioned segmentation set and high-precision map data to accurately obtain matching results.

[0066] Furthermore, different matching strategies can be selected based on actual conditions to meet different scenarios. For example, when lane marking types are clearly distinct, a lane marking type matching strategy can be prioritized. Similarly, when lane marking colors are clearly distinct, a lane marking color matching strategy can be prioritized. By combining multiple matching strategies, the uncertainty associated with a single strategy is eliminated.

[0067] It should be noted that the segmentation result obtained after panoramic segmentation includes the semantic segmentation results of all elements in the current frame image, including but not limited to the drivable area, shoulder segmentation results, and lane line segmentation results.

[0068] Step S240 , performing lane line point cloud matching based on the lane where the vehicle is located, to achieve lateral positioning of the vehicle position and correction of the heading angle.

[0069] After calculating and determining the vehicle's lane, lane line point cloud matching is performed based on the vehicle's lane to achieve lateral positioning of the vehicle and correction of the vehicle's heading angle. In other words, after lane selection, the point cloud matching algorithm can be used to match lane lines and correct the vehicle's lateral position and heading angle.

[0070] The lane line positioning method described above can reduce the cost of manually marking lane lines. At the same time, the results of image segmentation are more accurately matched with the prior knowledge in the high-precision map, thereby improving the accuracy of lane line recognition.

[0071] In one embodiment of the present application, the determination of the lane in which the vehicle is located may be based on at least one of a plurality of matching strategies, the plurality of matching strategies including: a matching strategy based on lane color, a matching strategy based on lane topology, and a matching strategy based on lane change judgment. The determination of the lane in which the vehicle is located includes: determining the lane in which the vehicle is currently located based on the confidence of the output results of the plurality of matching strategies; or, determining the lane in which the vehicle is currently located when two or more of the output results of the plurality of matching strategies are the same.

[0072] In specific implementation, the lane results of multiple lanes can usually be calculated according to the matching strategy, and then the current lane of the vehicle can be determined based on the confidence level of the output results of the multiple matching strategies (if there is only high confidence level).

[0073] In some embodiments, the current lane of the vehicle can be determined only when any two or more of the output results of the multiple matching strategies are the same (cannot be directly determined by confidence, but there are two or more results belonging to the same lane).

[0074] There is no need to pre-set weights for these matching strategies. Instead, each strategy is calculated and judged based on the actual situation. After selecting and determining which lane the vehicle is in, lateral correction can be performed based on the lane matching results.

[0075] In one embodiment of the present application, determining a lane of a vehicle based on a matching result of lane line information of the current road and lane line information in a segmentation result obtained by performing panoptic segmentation on an image of the current road includes:

[0076] According to the matching result of the lane line color in the lane line information, the color category of the lane line color in the lane line information of the current driving road is set; according to the number of color categories of the lane line color, the original pixel points in the lane line obtained after panoramic segmentation are clustered; according to the clustering result, the lane line obtained by panoramic segmentation is assigned a color; according to the color assignment result, the lane line is matched, and the color of the lane where the vehicle is currently located is calculated.

[0077] In specific implementations, based on the matching results of the lane line colors (yellow, white) in the lane line information, the color category of the lane line color in the lane line information of the current road is set (considering that in most scenarios, there will be two types, white and yellow, or only white). Then, based on the number of color categories of the lane line color (the number of groups corresponds to the number of color categories), the original pixel points in the lane lines obtained after panoptic segmentation are clustered. It should be noted that this is clustering of the original pixel points in the lane lines obtained after panoptic segmentation by color, which can usually be performed in HSV space.

[0078] Furthermore, based on the clustering results, the lane lines segmented by the panorama are assigned colors. After color assignment, lane line matching is facilitated, and finally the color of the lane where the vehicle is currently located is calculated, which can be represented by the parameter L_color in the specific lane line matching algorithm.

[0079] In one embodiment of the present application, the lane in which the vehicle is located is determined based on the matching result between the lane line information of the current driving road and the lane line information in the segmentation result obtained after panoramic segmentation of the image of the current driving road, including: calculating the rightmost lane based on the lane line topological relationship extracted from the high-precision map data and the drivable area, shoulder and lane line in the segmentation result obtained after panoramic segmentation; if the drivable area in the image includes the drivable area of ​​the entire road, calculating the distance between the vehicle and the edge of the rightmost lane; based on the distance between the vehicle and the edge of the rightmost lane, the lane in which the vehicle is currently located.

[0080] In specific implementations, the rightmost lane is calculated based on the lane topology extracted from the HD map data (the topology is consistent with the actual lanes, such as when there are multiple lanes going the same direction) and the segmentation results (pixel areas after semantic segmentation) obtained through panoptic segmentation, including the drivable area, shoulder segmentation, and lane line segmentation results. The vehicle's rightmost lane is calculated based on the segmented drivable area, shoulder, and lane line, combined with the lane topology extracted from the HD map. If the image contains the entire drivable area of ​​the road, the vehicle's lane is calculated based on the matching relationship of the rightmost lane line. This is represented by the parameter L_freespace in the lane matching algorithm.

[0081] It can be understood that the distance between the vehicle and the rightmost lane edge may be the road shoulder or the rightmost lane line.

[0082] In some embodiments, whether the image includes the entire drivable area of ​​the road can be determined based on the shape of the drivable area. If it does, the rightmost edge or shoulder of the drivable area will point to the vanishing point, thereby determining whether the image includes the entire drivable area of ​​the road.

[0083] It should be noted that vehicles drive on the right side of the road, and the rightmost lane line usually has a low detection rate (missed detection or false detection) due to obstructions, wear, and other factors. In this case, the rightmost lane can be used. If the current rightmost lane can be calculated, the vehicle's actual lane can be determined based on the correlation between lane lines.

[0084] In addition, the matching method of the lane type in the lane line information may be affected by occlusion, which may affect the detection results. In one embodiment of the present application, the method further includes: if the vehicle is equipped with a dynamic object detection module, using the detection results of the dynamic object detection module to process the lane line in the segmentation result obtained after panoramic segmentation to determine whether the lane line type is a dashed line; if the lane line type is a dashed line, matching the dashed lane line type in the lane line information of the current driving road with the segmentation result obtained after panoramic segmentation of the image of the current driving road.

[0085] At this point, considering that lane type matching calculations are affected by dynamic objects, as mentioned above, an additional dynamic object detection model is required, which increases computing power requirements. If the vehicle is equipped with a dynamic object detection module, the detection results of this dynamic object detection module are used to process the lane lines obtained after panoptic segmentation to determine whether the lane line type is dashed or solid. It is understood that considering computing power, lane type matching can be used as an optional condition.

[0086] In one embodiment of the present application, the panoramic segmentation of the lane in which the vehicle is located is determined based on the matching result of the lane line information of the current driving road and the lane line information in the segmentation result obtained after panoramic segmentation of the image of the current driving road, and also includes: determining the lane information of the vehicle at the previous moment and the heading angle change rate information of the vehicle based on the matching result of the lane line information, and judging whether the vehicle is performing a lane change operation at the current moment; matching the lane of the vehicle at the current moment according to the lane change operation judgment result, and determining the current lane of the vehicle.

[0087] In specific implementation, it is also possible to determine whether the vehicle is at least performing a lane change operation at the current moment based on the matching results of the lane line information, the lane information at the previous moment, and the vehicle's heading angle change rate Yawrate information (this information can indicate whether the vehicle has turned). Then, the lane in which the vehicle is currently located is matched according to the lane change operation results to determine the current lane of the vehicle.

[0088] In some embodiments, the vehicle may maintain its lane without changing lanes, and the current lane of the vehicle is determined to be the original lane.

[0089] In some embodiments, the vehicle may not maintain the lane and may change lanes, and the lane after the vehicle changes lanes is determined to be the lane after the lane change.

[0090] It can be understood that the above-mentioned matching processes of lane color, lane type and whether to change lanes are all calculated in real time and synchronously, and the lane determination result is obtained after the conditions are met.

[0091] The above method uses high-precision map data as a priori conditions to match the lane lines on the current road (the number of lane line colors on the current road will be provided during matching to determine the color category matching of the lane lines after segmentation), and combines the lane lines after panoramic segmentation and the drivable area information to locate the vehicle on the road, and finally uses the ICP point cloud matching method for lateral position correction. Taking into account that when the GNSS signal is interfered with, the positioning result will produce a large error, causing the positioning of the autonomous driving vehicle to deviate from the lane or even deviate from the road. Therefore, the method in this application is used to perform lateral position correction on the vehicle position based on the lane lines.

[0092] The embodiment of the present application also provides a lane line positioning device 200 for autonomous driving, such as Figure 2 As shown, a schematic diagram of the structure of a lane line positioning device for autonomous driving in an embodiment of the present application is provided. The lane line positioning device 200 for autonomous driving includes at least: an extraction module 210, a matching module 220, a lane determination module 230, and a positioning correction module 240, wherein:

[0093] In one embodiment of the present application, the extraction module 210 is specifically used to extract lane line information of the current driving road from the high-precision map data based on the positioning information provided by the vehicle combined navigation module and the current driving direction of the vehicle, and the lane line information includes at least lane line color, lane line type, and lane topology relationship.

[0094] It's understandable that using a lane-line-based lateral positioning correction method (algorithm) can ensure that autonomous vehicles stay within their lanes. However, this method (algorithm) is only effective if the positioning information obtained by the vehicle's integrated navigation module, IMU+GPS / RTK, is usable or the error during track derivation can be adjusted.

[0095] In one embodiment of the present application, before the lateral positioning correction of the vehicle position, it also includes: when the RTK state in the vehicle's integrated navigation module is a fixed solution, calculating the lateral correction error parameter of the current lane line matching algorithm, and when the RTK state is a fixed solution, the positioning state of the vehicle's integrated navigation module is an available state; when the positioning state of the vehicle's current integrated navigation module is an available state and is within a preset track deduction time threshold range, starting the lateral positioning correction of the vehicle position based on lane line matching according to the lateral correction error parameter.

[0096] In specific implementations, when the RTK state is a fixed solution, the positioning information of the vehicle integrated navigation module is available. At this point, the lateral correction error of the current lane matching algorithm is calculated and updated based on the actual situation. In other words, the lateral correction error err is introduced to adapt to the camera's internal and external parameters.

[0097] Furthermore, based on the statistical results of actual tests (historical data), the time threshold Thre_dr of track deduction is calculated. That is, the error of the track deduction of the IMU in the vehicle integrated navigation module can be corrected by the lane matching algorithm within a certain period of time, and lane matching is performed based on this error range.

[0098] It is understood that if the error is exceeded, it is considered uncorrectable and the positioning information is outdated.

[0099] It's important to note that when using track derivation, accumulated errors can cause the vehicle's trajectory to deviate significantly and erratically after exceeding a threshold, which cannot be corrected. Furthermore, with RTK using single-point or floating-point solutions, while the vehicle may experience meter-level deviations, this can be corrected if the autonomous vehicle remains on track or deviates slightly from the road.

[0100] Finally, it is necessary to start the lane matching algorithm to calculate lateral correction based on the current positioning status of the vehicle integrated navigation module (within the preset error range or positioning accuracy is accurate to the centimeter / millimeter level) or according to actual needs. It can be understood that when the positioning status of the vehicle integrated navigation module is available, that is, the RTK status is a fixed solution, no lateral correction is required. When the positioning status of the vehicle integrated navigation module is unavailable, lateral correction is required.

[0101] At this point, the lane matching algorithm has been successfully initialized and the lateral correction error err has been calculated. The lateral correction error err will no longer be updated subsequently, and the lateral correction function needs to be used continuously.

[0102] After the aforementioned pre-judgment, all lane line information of the current driving road can be extracted from the high-precision map data based on the positioning information in the vehicle integrated navigation module and the current driving direction of the vehicle.

[0103] In this case, the HD map data is pre-collected and generated within the preset driving area. This means that the integrated navigation module receives the positioning information received by the current integrated navigation module and, based on this positioning information and the vehicle's driving direction, extracts all lane marking information for the current lane in the HD map data.

[0104] For example, in order to prevent the vehicle from deviating into the opposite lane in combination with the vehicle's driving direction, all lane line information corresponding to the current driving road based on positioning information (centimeter / millimeter level positioning accuracy) is extracted from the high-precision map data. This lane line information includes the type and color of the lane line. The types of lane lines mainly include but are not limited to solid lines and dashed lines. The colors of lane lines mainly include but are not limited to white and yellow. In normal scenarios, solid lines, dashed lines, white lines, and yellow lines will all appear.

[0105] In one embodiment of the present application, the matching module 220 is specifically used to match the lane line information of the current driving road with the segmentation result obtained after panoramic segmentation of the image of the current driving road, and the image of the current driving road is obtained according to the vehicle visual perception module.

[0106] Using all lane line information of the current road as a priori conditions, the lane line information of the current road obtained after panoptic segmentation is matched. In other words, for each frame of the current road image, the panoptic segmentation results are used to match the lane line information. As mentioned above, the lane line information here may include different lane line types and colors, such as solid, dashed, white, and yellow.

[0107] By using the lane line data of the high-precision map as a priori conditions after panoramic segmentation, the type and color of the lane lines are calculated and matched, avoiding the lack of information in the pure semantic segmentation model or the incorrect color classification of the semantic recognition model.

[0108] Exemplarily, a panoramic segmentation result is obtained for each frame of the image, and the obtained segmentation result is combined with all lane line information of the current driving road as a priori conditions, and the lane line information in the segmentation result is matched and calculated again to obtain the corresponding lane line color and color number, lane line type and type number.

[0109] It should be noted that the segmentation result obtained after panoramic segmentation includes the semantic segmentation results of all elements in the current frame image, including the segmentation results for lane lines.

[0110] In one embodiment of the present application, the lane determination module 230 is specifically used to determine the lane in which the vehicle is located based on the matching result of the lane line information of the current driving road and the panoramic segmentation of the lane line information in the segmentation result obtained after panoramic segmentation of the image of the current driving road.

[0111] Based on the lane matching results, the vehicle's lane is calculated and determined based on the segmentation results obtained after panoptic segmentation. This means that the vehicle's current lane can be determined by combining the different segmentation results obtained after panoptic segmentation with the lane matching results. By using high-precision map data and the drivable area and road shoulder obtained after panoptic segmentation as prior information, the manual annotation cost for lane identification and classification is greatly reduced. The lane identification process utilizes the aforementioned segmentation set and high-precision map data to accurately obtain matching results.

[0112] Furthermore, different matching strategies can be selected based on actual conditions to meet different scenarios. For example, when lane marking types are clearly distinct, a lane marking type matching strategy can be prioritized. Similarly, when lane marking colors are clearly distinct, a lane marking color matching strategy can be prioritized. By combining multiple matching strategies, the uncertainty associated with a single strategy is eliminated.

[0113] It should be noted that the segmentation result obtained after panoramic segmentation includes the semantic segmentation results of all elements in the current frame image, including but not limited to the drivable area, shoulder segmentation results, and lane line segmentation results.

[0114] In one embodiment of the present application, the positioning correction module 240 is specifically used to perform lane line point cloud matching based on the lane where the vehicle is located, so as to achieve lateral positioning and heading angle correction of the vehicle position.

[0115] After the vehicle's lane is determined, lane line point cloud matching is performed based on the vehicle's lane to achieve lateral positioning of the vehicle and correction of the heading angle. That is, after lane selection, the point cloud matching algorithm can be used to perform lane line matching to correct the vehicle's lateral position and YAW heading angle.

[0116] The lane line positioning device described above can reduce the cost of manually marking lane lines. At the same time, the results of image segmentation can be more accurately matched with the prior knowledge in the high-precision map, thereby improving the accuracy of lane line recognition.

[0117] It can be understood that the above-mentioned lane line positioning device for autonomous driving can implement the various steps of the lane line positioning method for autonomous driving provided in the aforementioned embodiments. The relevant explanations about the lane line positioning method for autonomous driving are applicable to the lane line positioning device for autonomous driving and will not be repeated here.

[0118] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 3 At the hardware level, the electronic device includes a processor and, optionally, an internal bus, a network interface, and memory. The memory may include internal memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for its services.

[0119] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0120] The memory is used to store programs. Specifically, the program may include program code, which includes computer operating instructions. The memory may include internal memory and non-volatile memory, and provides instructions and data to the processor.

[0121] The processor reads the corresponding computer program from the non-volatile memory into the internal memory and then runs it, forming a lane line positioning device for autonomous driving at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:

[0122] Extracting lane line information of the current road from the high-precision map data based on the positioning information provided by the vehicle integrated navigation module and the current driving direction of the vehicle, wherein the lane line information includes at least lane line color, lane line type, and lane topology relationship;

[0123] matching lane line information of the current traveling road with a segmentation result obtained by performing panoptic segmentation on an image of the current traveling road, the image of the current traveling road being obtained according to a vehicle visual perception module;

[0124] determining a lane in which the vehicle is located based on a matching result of the lane line information of the current traveling road and the lane line information in the segmentation result obtained by performing panoptic segmentation on the image of the current traveling road;

[0125] Lane line point cloud matching is performed according to the lane where the vehicle is located to achieve lateral positioning of the vehicle position and correction of the heading angle.

[0126] The above application Figure 1The methods performed by the lane line positioning device for autonomous driving disclosed in the illustrated embodiments can be applied to or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in the processor or by software instructions. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in conjunction with the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method.

[0127] The electronic device may also perform Figure 1 A method for executing a lane line positioning device for autonomous driving in a vehicle lane line positioning device Figure 1 The functions of the illustrated embodiment will not be described in detail in the embodiments of the present application.

[0128] The embodiment of the present application also provides a computer-readable storage medium, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by an electronic device including multiple application programs, can enable the electronic device to execute Figure 1 The method performed by the lane line positioning device for autonomous driving in the illustrated embodiment is specifically used to perform:

[0129] Extracting lane line information of the current road from the high-precision map data based on the positioning information provided by the vehicle integrated navigation module and the current driving direction of the vehicle, wherein the lane line information includes at least lane line color, lane line type, and lane topology relationship;

[0130] matching lane line information of the current traveling road with a segmentation result obtained by performing panoptic segmentation on an image of the current traveling road, the image of the current traveling road being obtained according to a vehicle visual perception module;

[0131] determining a lane in which the vehicle is located based on a matching result of the lane line information of the current traveling road and the lane line information in the segmentation result obtained by performing panoptic segmentation on the image of the current traveling road;

[0132] Lane line point cloud matching is performed according to the lane where the vehicle is located to achieve lateral positioning of the vehicle position and correction of the heading angle.

[0133] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0134] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0135] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0136] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0137] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0138] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0139] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0140] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0141] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0142] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A lane line positioning method, wherein: The method comprises: Extracting lane line information of the current road from the high-precision map data based on the positioning information provided by the vehicle integrated navigation module and the current driving direction of the vehicle, wherein the lane line information includes at least lane line color, lane line type, and lane topology relationship; matching lane line information of the current traveling road with a segmentation result obtained by performing panoptic segmentation on an image of the current traveling road, the image of the current traveling road being obtained according to a vehicle visual perception module; determining a lane in which the vehicle is located based on a matching result of the lane line information of the current traveling road and the lane line information in the segmentation result obtained by performing panoptic segmentation on the image of the current traveling road; Determining a lane of the vehicle based on a matching result of the lane line information of the current traveling road and the lane line information in a segmentation result obtained by performing panoptic segmentation on the image of the current traveling road, including: Setting a color category of the lane line color in the lane line information of the current driving road according to a yellow or white matching result of the lane line color in the lane line information; Clustering original pixel points in the lane line obtained by panoptic segmentation according to the number of color categories of the lane line color; According to the clustering results, the lane lines segmented by the panorama are assigned colors; Match the lane lines according to the color assignment results and calculate the color of the lane where the vehicle is currently located; Lane line point cloud matching is performed according to the lane where the vehicle is located to achieve lateral positioning of the vehicle position and correction of the heading angle.

2. The method according to claim 1, wherein: The determining of the lane in which the vehicle is located may be based on at least one of a plurality of matching strategies, including: a matching strategy based on lane color, a matching strategy based on lane topology, and a matching strategy based on lane change judgment. The determining of the lane in which the vehicle is located includes: Determining the current lane of the vehicle based on the confidence levels of the output results of the multiple matching strategies; Alternatively, when two or more output results of the multiple matching strategies are the same, the lane in which the vehicle is currently located is determined.

3. The method according to claim 1, wherein: Determining a lane of the vehicle based on a matching result between lane line information of the current traveling road and lane line information in a segmentation result obtained by performing panoptic segmentation on the image of the current traveling road includes: Calculating the rightmost lane based on the lane line topology extracted from the high-precision map data and the drivable area, road shoulder, and lane lines in the segmentation result obtained after the panoptic segmentation; If the drivable area in the image contains the drivable area of ​​the entire road, the distance from the vehicle to the edge of the rightmost lane is calculated; The lane in which the vehicle is currently located is determined based on the distance between the vehicle and the edge of the rightmost lane.

4. The method according to claim 1, wherein: Determining the panoramic segmentation of the lane in which the vehicle is located based on a matching result between the lane line information of the current traveling road and the lane line information in the segmentation result obtained by performing panoramic segmentation on the image of the current traveling road, further comprising: Determining the lane information of the vehicle at the previous moment and the vehicle's heading angle change rate information based on the matching result of the lane line information, and judging whether the vehicle is currently performing a lane change operation; The lane of the vehicle at the current moment is matched according to the lane change operation judgment result to determine the current lane of the vehicle.

5. The method of claim 1, wherein: The method further comprises: If the vehicle is equipped with a dynamic object detection module, the detection result of the dynamic object detection module is used to process the lane line in the segmentation result obtained after the panoramic segmentation to determine whether the lane line type is a dotted line; In a case where the lane line type is a dotted line, the lane line type that is a dotted line in the lane line information of the current driving road is matched with a segmentation result obtained by performing panoptic segmentation on the image of the current driving road.

6. The method of claim 1, wherein: Before extracting all lane line information of the current driving road from the high-precision map data based on the positioning information provided by the vehicle integrated navigation module and the current driving direction of the vehicle, the method further includes: When the RTK in the vehicle integrated navigation module is a fixed solution, calculating a lateral correction error parameter of the current lane line matching algorithm, and when the RTK is a fixed solution, the positioning state of the vehicle integrated navigation module is an available state; When the positioning status of the vehicle's current integrated navigation module is available and within the preset track deduction time threshold, the lateral positioning and heading angle correction of the vehicle position based on lane line matching is enabled according to the lateral correction error parameter.

7. A lane line positioning device, wherein: The device comprises: An extraction module is used to extract lane line information of the current road from the high-precision map data based on the positioning information provided by the vehicle integrated navigation module and the current driving direction of the vehicle. The lane line information includes at least lane line color, lane line type, and lane topology relationship; a matching module, configured to match lane line information of the current traveling road with a segmentation result obtained by performing panoptic segmentation on an image of the current traveling road, the image of the current traveling road being obtained according to the vehicle visual perception module; a lane determination module, configured to determine a lane in which a vehicle is located based on a matching result of the lane line information of the current traveling road and the lane line information in the segmentation result obtained by performing panoptic segmentation on the image of the current traveling road; Determining a lane of the vehicle based on a matching result of the lane line information of the current traveling road and the lane line information in a segmentation result obtained by performing panoptic segmentation on the image of the current traveling road, including: Setting a color category of the lane line color in the lane line information of the current driving road according to a yellow or white matching result of the lane line color in the lane line information; Clustering original pixel points in the lane line obtained by panoptic segmentation according to the number of color categories of the lane line color; According to the clustering results, the lane lines segmented by the panorama are assigned colors; Match the lane lines according to the color assignment results and calculate the color of the lane where the vehicle is currently located; The positioning correction module is used to perform lane line point cloud matching based on the lane where the vehicle is located, so as to realize the lateral positioning of the vehicle position and the correction of the heading angle.

8. An electronic device comprising: processor; as well as A memory arranged to store computer executable instructions, which when executed cause the processor to perform the method of any one of claims 1 to 6.

9. A computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of application programs, causes the electronic device to execute the method according to any one of claims 1 to 6.

Images