Obstacle perception method and apparatus for autonomous vehicle

Abstract

The application discloses a kind of obstacle perception method, device for automatic driving vehicle, the method is executed on automatic driving vehicle, it includes according to the camera perception result on automatic driving vehicle, the position of obstacle in image coordinate system is judged, and 2D lane positioning result is obtained;According to the multi-sensor perception fusion result on the automatic driving vehicle, the position of the obstacle in the body coordinate system is judged, and 3D lane positioning result is obtained;According to the 2D lane positioning result and the 3D lane positioning result, the lane where the obstacle is located is determined.By the method of 2D pixel and 3D scene combination in the application, it is identified which lane the obstacle is in, and the accuracy of the self-vehicle obstacle perception result is improved.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to an obstacle perception method and apparatus for autonomous vehicles. Background Technology

[0002] Accurately perceiving all obstacles around the vehicle and identifying the lane where the obstacle is located is of great significance for the decision-making and planning module of the autonomous driving system to rationally plan the vehicle's driving route and avoid collision risks.

[0003] Among related technologies, the accuracy of autonomous vehicles in perceiving which lane each obstacle in their vicinity is relatively poor. Summary of the Invention

[0004] This application provides an obstacle perception method and apparatus for autonomous vehicles to improve obstacle perception accuracy.

[0005] The embodiments of this application adopt the following technical solutions:

[0006] In a first aspect, embodiments of this application provide an obstacle perception method for autonomous vehicles, wherein the method includes:

[0007] Based on the camera perception results on the autonomous vehicle, the position of the obstacle in the image coordinate system is determined, and the 2D lane localization result is obtained.

[0008] Based on the multi-sensor perception fusion results on the autonomous vehicle, the position of the obstacle in the vehicle coordinate system is determined, and the 3D lane positioning result is obtained.

[0009] Based on the 2D lane positioning results and the 3D lane positioning results, the lane where the obstacle is located is determined.

[0010] In some embodiments, the method further includes:

[0011] Pre-acquire lane information from high-precision map data of the current driving segment of the autonomous vehicle;

[0012] The lane line information from the camera perception results on the autonomous vehicle is fused with the lane information in the high-precision map data and then correlated and tracked to obtain lane structure information based on the lane lines in the high-precision map and the lane lines perceived by the camera. The lane structure information includes at least lane ID and lane line ID.

[0013] In some embodiments, determining the position of the obstacle in the image coordinate system based on the camera perception results on the autonomous vehicle to obtain the 2D lane positioning result includes:

[0014] Based on the camera perception results on the autonomous vehicle, obtain the curve parameters of the perceived lane line in the image coordinate system and the coordinates of the four corner points of the obstacle's envelope in the image coordinate system on the same image frame;

[0015] Based on the curve parameters and the coordinates of the four corner points of the envelope, the accurate position of the obstacle in the image coordinate system is determined, and the 2D lane positioning result is obtained.

[0016] In some embodiments, determining the accurate position of the obstacle in the image coordinate system based on the curve parameters and the coordinates of the four corner points of the envelope to obtain a 2D lane positioning result includes:

[0017] If the lane lines are not obscured by the obstacle, all lanes in the lane structure are traversed according to the lane structure information, and the proportional relationship between the obstacle and the left and right lane lines of one of the lanes is calculated.

[0018] The lane in which the obstacle is located is determined based on the proportional relationship between the obstacle and the left and right lane lines of one of the lanes.

[0019] If both the lower left and lower right points of the obstacle are inside a lane, then the obstacle is in the current lane.

[0020] If the obstacle spans two lanes, the proportion of the obstacle's width in the two lanes is calculated, and it is determined whether the obstacle is in the current lane or spans two lanes.

[0021] In some embodiments, determining the accurate position of the obstacle in the image coordinate system based on the curve parameters and the coordinates of the four corner points of the envelope to obtain a 2D lane positioning result further includes:

[0022] Calculate the horizontal positional relationship between the bottom left and bottom right points of the obstacle's bounding box and the left and right lane lines of each lane, and the ratio of the bounding box width to bottom width. The left and right lane lines are the horizontal positions of the lane lines in the image coordinate system.

[0023] If the obstacle is located in the lane where the current autonomous vehicle is traveling, then both left_over and right_over in the preset proportional relationship are positive values;

[0024] If the obstacle is located in the right lane of the vehicle currently traveling, then the right_over value is negative;

[0025] If the obstacle is located in the left lane of the vehicle currently traveling, then right_over is a positive value;

[0026] The obstacle is located in which lane ID, and the 2D lane positioning result calculated on the 2D pixel plane is assigned a first confidence level.

[0027] In some embodiments, determining the accurate position of the obstacle in the image coordinate system based on the curve parameters and the coordinates of the four corner points of the envelope to obtain a 2D lane positioning result further includes:

[0028] When the obstacle spans two lanes, and one lane line is obscured by the obstacle and is therefore an invalid lane line:

[0029] If the left lane line of a lane is invalid, then use right_over;

[0030] If right_over is negative, the obstacle is located in the right lane of the current lane;

[0031] If right_over is a positive value, then the obstacle is located in the current lane or the left lane of the current lane;

[0032] The case where the right_over is negative and the obstacle is located in the right lane ID of the current lane is recorded as the detection result, and the 2D lane positioning result calculated on the 2D pixel plane is assigned a first confidence level.

[0033] In some embodiments, the method further includes:

[0034] When the obstacle is incomplete in the image:

[0035] If the obstacle is determined to be in a truncated state based on the truncated flag of the image perception result of the autonomous vehicle, then the lane in which the truncated obstacle is located is calculated based on the obtained truncated obstacle.

[0036] Based on the 2D lane positioning result calculated on the 2D pixel plane of the truncated obstacle, a second confidence level is assigned, which is less than the first confidence level.

[0037] In some embodiments, determining the position of the obstacle in the vehicle coordinate system based on the multi-sensor perception fusion results on the autonomous vehicle to obtain 3D lane positioning results includes:

[0038] In the vehicle coordinate system of an autonomous vehicle, the 3D envelope data, center point coordinates, and lane line information of the obstacle are obtained after the fusion of multi-sensor data.

[0039] Based on the relative positions of the four corner points on the bottom surface of the 3D envelope and the lane lines, determine which lane the obstacle is in;

[0040] If the obstacle is in one lane, then that lane is recorded; if the obstacle spans two lanes, then both lanes are recorded.

[0041] Based on the lane recording results, the 3D lane positioning results calculated in 3D space will be assigned a third confidence level.

[0042] In some embodiments, determining the lane where the obstacle is located based on the 2D lane positioning result and the 3D lane positioning result includes:

[0043] The confidence levels of the 2D lane positioning results and the 3D lane positioning results are compared, and the result with the higher confidence level is taken as the final result to determine the lane where the obstacle is located.

[0044] Secondly, embodiments of this application also provide an obstacle sensing device for autonomous vehicles, wherein the device includes:

[0045] The 2D pixel plane positioning module is used to determine the position of obstacles in the image coordinate system based on the camera perception results on the autonomous vehicle, and obtain 2D lane positioning results.

[0046] The 3D spatial positioning module is used to determine the position of the obstacle in the vehicle coordinate system based on the multi-sensor perception fusion results on the autonomous vehicle, and obtain the 3D lane positioning result.

[0047] The determination module is used to determine the lane where the obstacle is located based on the 2D lane positioning results and the 3D lane positioning results.

[0048] Thirdly, embodiments of this application also provide an electronic device, including: a processor; and a memory arranged to store computer-executable instructions, which, when executed, cause the processor to perform the above-described method.

[0049] Fourthly, embodiments of this application also provide a computer-readable storage medium that stores one or more programs, which, when executed by an electronic device including multiple applications, cause the electronic device to perform the above-described method.

[0050] The at least one technical solution adopted in this application embodiment can achieve the following beneficial effects: First, based on the camera perception results on the autonomous vehicle, the position of the obstacle in the image coordinate system is determined to obtain a 2D lane positioning result. Then, based on the multi-sensor perception fusion results on the autonomous vehicle, the position of the obstacle in the vehicle coordinate system is determined to obtain a 3D lane positioning result. Finally, based on the 2D lane positioning result and the 3D lane positioning result, the lane where the obstacle is located is determined. That is to say, when lane lines are lost, lane lines from a high-precision map can be used to determine which lane the obstacle is in, or in scenarios without a high-precision map, perceived lane lines can be used to determine which lane the obstacle is in. In addition, the clear and definite relative positions of obstacles and lane lines in 2D images can improve the accuracy of perception; or when obstacles are partially occluded or abnormal in 2D images, the 3D lane positioning result can be used to assist in determining which lane the obstacle is in by using the envelope size tracked by the rear and lateral sensors. Attached Figure Description

[0051] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0052] Figure 1 This is a flowchart illustrating an obstacle perception method for autonomous vehicles in an embodiment of this application.

[0053] Figure 2 This is a schematic diagram illustrating the implementation principle of the obstacle perception method for autonomous vehicles in the embodiments of this application;

[0054] Figure 3 The lane ID and lane line ID are used in the obstacle perception method for autonomous vehicles in the embodiments of this application;

[0055] Figure 4 This is a schematic diagram illustrating the situation where an obstacle is located in a certain lane in an embodiment of this application;

[0056] Figure 5 This is a schematic diagram illustrating a scenario where an obstacle crosses two lanes in an embodiment of this application;

[0057] Figure 6 This is a schematic diagram illustrating the situation where the obstacle is cut off in an embodiment of this application;

[0058] Figure 7 This is a schematic diagram of the obstacle perception device structure for autonomous vehicles in an embodiment of this application;

[0059] Figure 8 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0061] During their research, the inventors discovered that, in related technologies, the main methods for autonomous vehicles to determine which lane each obstacle in their vicinity belongs to include the following:

[0062] Method 1 combines the coordinates of the obstacle's center point in the vehicle's coordinate system with the lane line curve equation given by the perception to determine the relationship between the obstacle and the lane line, and then determine the lane where the obstacle is located.

[0063] Method 2 does not use lane lines; instead, it projects the map data of the area to be measured ahead into the image data and finds the two lane lines with the smallest distance to the obstacle bounding box in the image domain, thus obtaining the lane where the obstacle is located.

[0064] Method 3: Obtain the coordinates of the lane lines from the high-precision map, construct the Frenet road coordinate system based on the lane centerline, fit the curve equations of the left and right lane lines in the Frenet coordinate system, construct the approximate range of each pair of lane lines, and determine whether the vehicle is within the constraint range of a certain pair of lanes to determine the lane where the obstacle is located.

[0065] Method 4: Based on the overlap between the obstacle and its enclosure (envelope) detected by the camera and other sensors and the lane and its lane mask on the driving surface, select the lane with the highest overlap rate and consider it to be the lane where the obstacle is located.

[0066] Based on further research, the shortcomings of Method 1 are as follows: using the coordinates of the obstacle's center point in the vehicle's coordinate system provided by perception can only indicate the lane in which the obstacle's center point is located. If the obstacle crosses a lane, it will cause inaccurate judgment. For example, if the obstacle changes lanes and the vehicle body has already crossed the lane, but the center point is still in the original lane, it will lead to inaccurate judgment. In addition, the effective distance of the perceived lane lines is limited. When the obstacle exceeds the limited range of the perceived lane lines, it is impossible to determine the lane in which the obstacle is located. Furthermore, when there is a large vehicle traveling ahead, the perceived lane lines may be obscured, and the lane lines between the left and right lanes will be lost in the perception results, making it impossible to accurately locate the obstacle's lane.

[0067] The drawbacks of methods 2 and 3 are: they rely heavily on high-precision maps, and the functionality will fail in scenarios without lane-level high-precision maps. Furthermore, when projecting high-precision map data into the vehicle coordinate system in real time, it depends on the vehicle's positioning information and heading angle, leading to accumulated errors and poor accuracy. For distant targets, the camera's ranging accuracy drops significantly, and converting the obstacle perception from the camera to the high-precision map in this case results in substantial errors.

[0068] The drawbacks of Method 4 are: a. It only applies to lane identification for obstacles within the drivable area in front of the vehicle; it cannot be used to determine the lane of obstacles on the sides or behind the vehicle. b. It only determines the lane of an obstacle on the pixel plane; when an obstacle is occluded or only part of it is identified, the obstacle's object fence is incomplete, and the correct lane assignment cannot be given.

[0069] To address the aforementioned shortcomings, the obstacle perception method for autonomous vehicles in this application embodiment can use lane lines from a high-precision map to determine which lane the obstacle is in when lane lines are lost, and can also use perceived lane lines to determine which lane the obstacle is in in scenarios where there is no high-precision map.

[0070] Furthermore, the method in this application can take advantage of the clear and distinct relative positions of obstacles and lane lines in 2D images, and can also determine which lane the obstacle is in by using the envelope size tracked by the rear and lateral sensors when the obstacle is partially occluded in the 2D image.

[0071] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0072] This application provides an obstacle perception method for autonomous vehicles, such as... Figure 1 The diagram shows a flowchart of an obstacle perception method for autonomous vehicles in an embodiment of this application. The method includes at least the following steps S110 to S130:

[0073] Step S110: Based on the camera perception results on the autonomous vehicle, determine the position of the obstacle in the image coordinate system to obtain the 2D lane positioning result.

[0074] To better determine the lane where the obstacle is located, lane information from the high-precision map data of the current driving road and lane line information from the camera on the autonomous vehicle can be obtained in advance. The two are then fused and correlated through multi-sensor data to provide a lane structure result that combines the lane lines from the high-precision map and the perceived lane lines. If either the perceived lane line or the high-precision map lane line is lost, no target data correlation and tracking is performed, and the lane line result from a single sensor can be directly issued.

[0075] Autonomous vehicles typically have multiple types of sensors, including LiDAR, millimeter-wave radar, and vision sensors, which are used to achieve multi-sensor fusion perception results.

[0076] For visual sensors, multiple cameras may be installed on the autonomous vehicle. The specific installation location of the cameras is not specifically limited in the embodiments of this application, and those skilled in the art can choose according to the actual situation. For the camera perception results on the autonomous vehicle (including at least one camera), the position of the obstacle in the image coordinate system is determined in the 2D image pixels, which is used as the 2D lane positioning result.

[0077] It is important to note that at this point, it is not necessary to determine whether there is any occlusion in the perception result; it is only necessary to calculate the confidence level of the perception result later.

[0078] Furthermore, since each frame of the camera image contains information about lane lines and obstacles (images without this information are not considered results), the results of lane line detection and obstacle detection in image perception can both be reflected in the same frame, such as... Figure 4 As shown, the curve parameters of the perceived lane lines in the image coordinate system (e.g., lane lines, traffic signs) and the coordinates of the four corner points of the target obstacle's bounding box in the image coordinate system can be obtained in the same frame.

[0079] Step S120: Based on the multi-sensor perception fusion results on the autonomous vehicle, determine the position of the obstacle in the vehicle coordinate system to obtain the 3D lane positioning result.

[0080] As mentioned earlier, autonomous vehicles include various types of sensors, and the position of the obstacle in the vehicle coordinate system can be determined based on the multi-sensor perception fusion results, thus obtaining the 3D lane positioning result in a three-dimensional scene.

[0081] Furthermore, in the current vehicle coordinate system, the 3D envelope data, center point coordinates, and lane line information of the obstacle are acquired after data fusion from multiple sensors (which could be cameras, LiDAR, or even more sensors). Based on the relative positions of the four corner points on the bottom surface of the 3D envelope with the lane lines, the lane in which the obstacle is located is determined. In other words, through the multi-sensor perception fusion results, it is possible to determine which lane the target obstacle (an obstacle appearing from in front of or behind the vehicle, or from the left or right side) is in. Because of the availability of high-precision map data, the lane in which the target obstacle is located can be determined based on the fused perception results.

[0082] For example, either visual SLAM or laser SLAM can be used to determine which specific lane an obstacle is in.

[0083] It is important to note that in 3D space, the accuracy of both obstacle location information obtained from multi-sensor data fusion and lane line information obtained from the fusion of perception and high-precision maps decreases as the distance increases.

[0084] Step S130: Determine the lane where the obstacle is located based on the 2D lane positioning result and the 3D lane positioning result.

[0085] Based on the 2D lane positioning results and the 3D lane positioning results obtained above, the confidence levels of the lane division results in 2D and 3D spaces are compared, and the result with the higher confidence level is taken as the final result.

[0086] If only the lane line results perceived by the camera are used, the effective distance is usually within 80-100 meters. When encountering curves, the effective distance of lane line perception will be even shorter. 3D lane positioning results can solve the problem that when the longitudinal position of an obstacle exceeds the effective range of the perceived lane line, there are no lane lines around, and it is impossible to determine which lane the obstacle is in based on its relative position.

[0087] Using the above method, based on 2D lane localization results, when the target obstacle is not obscured, the clear and unambiguous 2D lane localization results, utilizing the relative positions of the obstacle and lane lines in the 2D image, can be used as the obstacle perception result. Based on 3D lane localization results, when lane lines are lost, lane lines from a high-precision map can be used to determine which lane the obstacle is in. Simultaneously, in scenarios without a high-precision map, perceived lane lines can be used to determine which lane the obstacle is in.

[0088] Furthermore, by analyzing the lane location of obstacles in different scenarios, the specific lane or lane line where the obstacle is located can be obtained, covering all possible situations. That is, when the obstacle outline is fully exposed in the 2D image, the relative position of the lane line and the obstacle is accurate; however, when the obstacle just enters the field of view (FOV) of the visual sensor, the obstacle outline is incomplete, and the vision cannot determine which lane the obstacle is in. In this case, in 3D space, the tracking results of multiple sensors behind, to the left and right sides of the obstacle can be fused from multiple frames to comprehensively determine which lane the obstacle is in.

[0089] Using the above method, even in scenarios without lane-level high-precision maps, it is possible to accurately determine which lane an obstacle is located in. It can not only determine which lane an obstacle in the drivable area behind the vehicle is in, but also which lane an obstacle to the left or right of the autonomous vehicle is in.

[0090] Unlike related technologies that use a single type of sensor, this technology can determine the specific situation of an obstacle in a 2D image based on the obstacle's bounding box and corner coordinates when the obstacle is located in a lane, crosses two lanes, or is cut off or obscured.

[0091] Unlike related technologies where the perception results of autonomous vehicles heavily rely on high-definition maps, making it difficult to accurately determine which lane an obstacle is in when lane lines are missing, this approach uses lane lines from a high-definition map to determine the obstacle's lane location, and can also use perceived lane lines to determine the obstacle's lane location in scenarios without a high-definition map.

[0092] Unlike related technologies that can only determine which lane an obstacle in front of the vehicle belongs to, but cannot determine which lane obstacles to the left, right, or rear of the vehicle belong to, this technology can comprehensively determine the lane of an obstacle in 3D space by fusing multi-frame tracking results from multiple rear and side sensors.

[0093] In one embodiment of this application, the method further includes: acquiring lane information in high-precision map data of the current driving segment of the autonomous vehicle in advance; performing correlation tracking based on the lane line information in the camera perception results on the autonomous vehicle and the lane information in the high-precision map data after data fusion, to obtain lane structure information based on the lane lines in the high-precision map and the lane lines perceived by the camera, wherein the lane structure information includes at least lane ID and lane line ID.

[0094] like Figure 3 As shown, the lane structure results include the number of lanes on the currently traveling road. Each lane has a lane ID, and the current lane (the lane the vehicle is currently traveling in), left lane, right lane, etc., all have clearly identifiable IDs. Each lane consists of two lane lines, and the IDs of the left and right lane lines that make up that lane are also known. Based on the lane ID and lane line ID, the specific location of the obstacle within the lane can be determined.

[0095] In one embodiment of this application, the step of determining the position of an obstacle in the image coordinate system based on the camera perception results on the autonomous vehicle to obtain a 2D lane positioning result includes: obtaining the curve parameters of the perceived lane line in the image coordinate system and the coordinates of the four corner points of the envelope of the obstacle in the image coordinate system on the same image frame based on the camera perception results on the autonomous vehicle; determining the accurate position of the obstacle in the image coordinate system based on the curve parameters and the coordinates of the four corner points of the envelope to obtain a 2D lane positioning result.

[0096] like Figure 4As shown, each frame of the camera image contains information about lane lines and obstacles. The results of lane line detection and obstacle detection in image perception can be reflected in the same frame image. The curve parameters of the perceived lane line in the image coordinate system and the coordinates of the four corner points of the envelope of the obstacle in the image coordinate system are obtained in the same frame.

[0097] Furthermore, considering that obstacles may be within a single lane or cross two lanes during vehicle operation (e.g., when a vehicle ahead changes lanes), the autonomous vehicle's camera may capture the complete outline of the obstacle or only a portion of it. Therefore, the lane location result of the obstacle can be obtained in different scenarios, including when the obstacle is exactly in a single lane, crosses two lanes, or is cut off / obstructed.

[0098] In one embodiment of this application, determining the accurate position of the obstacle in the image coordinate system based on the curve parameters and the coordinates of the four corner points of the envelope to obtain a 2D lane positioning result includes: when the lane lines are not obscured by the obstacle, traversing all lanes in the lane structure according to the lane structure information and calculating the proportional relationship between the obstacle and the left and right lane lines of one of the lanes; determining which lane the obstacle is in based on the proportional relationship between the obstacle and the left and right lane lines of one of the lanes; if the lower left and lower right points of the obstacle are both inside a lane, then the obstacle is in the current lane; if the obstacle spans two lanes, calculating the ratio of the obstacle's width to the width of the two lanes and determining whether the obstacle is in the current lane or spans two lanes.

[0099] Category a of scenario 1: The lane lines are not obstructed, and the obstacle is entirely within one lane.

[0100] bottom_width=|bottom_right-bottom_left|

[0101] According to the calculation formula:

[0102] left_over=(bottom_right-left_pre) / bottom_width; left_over>1 in this column;

[0103] right_over=(bottom_left-right_pre) / bottom_width; right_over>1 in this column;

[0104] As shown above, the ratio of right_over to left_over is greater than 1, so the obstacle is located in one lane at this time.

[0105] In case 2, category b, the lane lines are not obstructed, and the obstacle spans two lanes (lane 1 and lane 2). According to the calculation formula,

[0106] For lane 1, {left_over>1, right_over<1}, the ratio could be 0.3 or 0.5, etc. The proportion of left_over within the lane can be determined based on the actual situation; if it exceeds the width of the obstacle by 0.3, it is considered to be within the lane.

[0107] For lane 2, {left_over<1,right_over>1}, the same as lane 1, determines whether it is within the lane based on the proportion occupied in that lane.

[0108] In summary, obstacles are not necessarily in the same lane. For example, when another vehicle (acting as an obstacle) cuts in or out, it crosses two lanes. The lane in which the obstacle is located is determined based on the proportion of the obstacle in the two lanes.

[0109] Traverse all lanes within the lane structure and calculate the distance relationship between the obstacle and the left and right lane lines of a given lane. Calculation formula:

[0110] left_over=(bottom_right-left_pre) / bottom_width

[0111] Right_over=(right_pre-bottom_left) / bottom_width

[0112] In other words, if the lower left and lower right points of the obstacle are both within a lane, then the obstacle is definitely within that lane. If the obstacle spans two lanes, calculate the ratio of the obstacle's width to the width of the two lanes.

[0113] For example, with a threshold of 0.3, if left_over = 0.2 and Right_over = 0.8, then it is only in the right lane; if left_over = 0.31 and Right_over = 0.79, then it is in both lanes; if left_over = 0.5 and Right_over = 0.5, then it is in both lanes; and if left_over = 0.7 and Right_over = 0.2, then it is only in the left lane.

[0114] Calculate the horizontal positional relationship between the bottom left and bottom right points of the obstacle's bounding box and the left and right lane lines of each lane, and the ratio of the bounding box width to bottom width. The left and right lane lines are the horizontal positions of the lane lines in the image coordinate system.

[0115] If the obstacle is located in the lane where the current autonomous vehicle is traveling, then both left_over and right_over in the preset proportional relationship are positive values;

[0116] If the obstacle is located in the right lane of the vehicle currently traveling, then the right_over value is negative;

[0117] If the obstacle is located in the left lane of the vehicle currently traveling, then right_over is a positive value;

[0118] The obstacle is located in which lane ID, and the 2D lane positioning result calculated on the 2D pixel plane is assigned a first confidence level.

[0119] In some embodiments, determining the accurate position of the obstacle in the image coordinate system based on the curve parameters and the coordinates of the four corner points of the envelope to obtain a 2D lane localization result further includes: when the obstacle crosses two lanes and one lane line is occluded by the obstacle and is an invalid lane line: when the left lane line of one lane is invalid, then right_over is used; if right_over is negative, then the obstacle is located in the right lane of the current lane; if right_over is positive, then the obstacle is located in the current lane or the left lane of the current lane; the case where right_over is negative and the obstacle is located in the right lane ID of the current lane is recorded as the detection result, and the 2D lane localization result calculated on the 2D pixel plane is assigned a first confidence level.

[0120] Lane IDs and lane line IDs can be obtained from high-precision map data or preset.

[0121] In practice, when an obstacle is only in one lane and the lane lines are not obstructed, [the following applies].

[0122] Calculate the lateral positional relationship between the bottom left and bottom right points of the obstacle envelope and the two lane lines (left_line[i] and right_line[i]) of each lane, and the ratio of the width (bottom_width) of the envelope:

[0123] left_over=(bottom_right-left_line[i]) / bottom_width;

[0124] right_over=(bottom_left-right_line[i]) / bottom_width;

[0125] If the obstacle is located in the lane the vehicle is currently traveling in, both its left_over and right_over values ​​are positive; if the obstacle is located in the right lane the vehicle is currently traveling in, its right_over value is negative; the same applies to the left lane. This allows us to determine which lane the obstacle is in. At this point, the result calculated on the 2D pixel plane is assigned a confidence level of 1.0.

[0126] In some embodiments, determining the accurate position of the obstacle in the image coordinate system based on the curve parameters and the coordinates of the four corner points of the envelope to obtain a 2D lane localization result further includes: when the obstacle crosses two lanes and one lane line is occluded by the obstacle and is an invalid lane line: when the left lane line of one lane is invalid, then right_over is used; if right_over is negative, then the obstacle is located in the right lane of the current lane; if right_over is positive, then the obstacle is located in the current lane or the left lane of the current lane; the case where right_over is negative and the obstacle is located in the right lane ID of the current lane is recorded as the detection result, and the 2D lane localization result calculated on the 2D pixel plane is assigned a first confidence level.

[0127] When an obstacle crosses two lanes, and one lane line is obscured by the obstacle, resulting in an invalid lane line (Situation Two):

[0128] Left_over and right_over can be calculated in the same way as in Case 1. If the left lane line of a certain lane is invalid, then only right_over can be used.

[0129] If right_over is negative, the obstacle is located in the right lane of the current lane;

[0130] If right_over is a positive value, the obstacle is located in the current lane or the lane to the left of the current lane;

[0131] It should be noted that, since the left lane line of a certain lane is invalid, only the case where right_over is negative and the obstacle is located in the right lane of the current lane is considered and recorded in the detection results. The result calculated on the 2D pixel plane is assigned a confidence level of 1.0.

[0132] In one embodiment of this application, the method further includes: when the obstacle is incomplete in the image: if the obstacle is determined to be in a truncated state based on the truncated flag of the image perception result of the autonomous vehicle, then the lane in which the truncated obstacle is located is calculated based on the obtained truncated obstacle; based on the 2D lane positioning result calculated on the 2D pixel plane of the truncated obstacle, a second confidence level is assigned, wherein the second confidence level is less than the first confidence level.

[0133] When the obstacle is incomplete in the image, in case three:

[0134] If the image perception result indicates that the obstacle is in a truncated state, the method of Case 2 or Case 1 is still used to calculate which lane it is in. Then, the lane localization result calculated on the 2D pixel plane of the truncated obstacle is assigned a confidence level of 0.2.

[0135] The final result in 2D pixels is compared with the result in 3D space, and the result with higher confidence is selected as the final result.

[0136] It should be noted that visual perception typically provides a flag indicating whether the data has been truncated, which can be obtained by those skilled in the art in various ways, and is not specifically limited in the embodiments of this application.

[0137] In one embodiment of this application, determining the position of the obstacle in the vehicle coordinate system based on the multi-sensor perception fusion result on the autonomous vehicle to obtain a 3D lane positioning result includes: acquiring the 3D envelope data, center point coordinates, and lane line information of the obstacle after multi-sensor data fusion in the vehicle coordinate system of the autonomous vehicle; determining which lane the obstacle is in based on the relative positions of the four corner points on the bottom surface of the 3D envelope and the lane lines; recording the lane if the obstacle is in one lane, and recording both lanes if the obstacle spans two lanes; and assigning a third confidence level to the 3D lane positioning result calculated in 3D space based on the lane recording result.

[0138] In the current vehicle coordinate system, acquire the 3D bounding box data, center point coordinates, and lane line information of the obstacle after data fusion from multiple sensors (camera, radar, or even more sensors). Based on the relative positions of the four corner points on the bottom surface of the 3D bounding box and the lane lines, determine which lane the obstacle is in. If the obstacle spans two lanes, record both lanes in the result.

[0139] Furthermore, when acquiring the 3D envelope data, center point coordinates, and lane line information of the obstacle after multi-sensor data fusion in the vehicle coordinate system of the autonomous vehicle, it also includes all situations of obstacles behind, to the left, and to the rear of the autonomous vehicle.

[0140] It is understandable that four corner points are used in 3D space while only two corner points are used in 2D space. This is because the 2D perception result only frames the visible surface of the obstacle, which is sufficient to use two corner points. However, the 3D frame is the actual envelope frame that is inferred, so four corner points are used in 3D space.

[0141] In 3D space, the accuracy of both obstacle location information from multi-sensor data fusion and lane line information from the fusion of perception and high-precision maps decreases with increasing distance. Therefore, in 3D space, the confidence level for lane line segmentation is set as follows:

[0142] Conf=0.2+(1-(x-x_min) / (x_max-x_min))*0.8

[0143] Where X is the ordinate of the vehicle's center point, x_max is the ordinate of the furthest position of the lane line, and x_min is the ordinate of the closest position of the lane line.

[0144] It's understandable that if lane lines are valid and the image is unobstructed, the confidence level in 2D space is higher than in 3D. However, if the image is obstructed or the perceived lane lines are invalid, the confidence level in 3D is higher than in 2D, because even when the perceived lane lines are obstructed or missed, the fused lane lines still exist, making the results in 3D space more accurate.

[0145] In one embodiment of this application, determining the lane where the obstacle is located based on the 2D lane positioning result and the 3D lane positioning result includes: comparing the confidence levels of the 2D lane positioning result and the 3D lane positioning result, and taking the result with the higher confidence level as the final result to determine the lane where the obstacle is located.

[0146] By combining 2D and 3D lane localization results, the lane in which an obstacle is located can be identified. Both 2D and 3D methods have their advantages and disadvantages, and they complement each other, resulting in more accurate results. Furthermore, when the obstacle's outline is fully exposed in the 2D image, the relative position of the lane lines and the obstacle is accurate. However, when the obstacle just enters the field of view (FOV) of the visual sensor, the obstacle's outline is incomplete, and the visual system cannot determine which lane the obstacle is in. In this case, in 3D space, multi-frame tracking results from rear and side sensors can be fused to comprehensively determine the obstacle's lane location.

[0147] Based on the above method, the lane lines in the camera perception results and the high-precision lane lines in the high-precision map of the current driving road obtained in advance can be used to determine which lane the obstacle is in if one is missing. If both are present, a more accurate lane structure can be given, which helps to more accurately determine which lane the obstacle is in.

[0148] Please refer to Figure 2 This is a schematic diagram illustrating the implementation principle of the obstacle perception method for autonomous vehicles in this application embodiment. The specific implementation steps are as follows:

[0149] like Figure 3 As shown, the lane structure results include how many lanes are on the road currently in use. Each lane has a lane ID. The current lane (the lane the vehicle is currently traveling in), the left lane, the right lane, etc., all have clearly identifiable IDs. Each lane consists of two lane lines, and the IDs of the left and right lane lines that make up the lane are known.

[0150] Step 1: Obtain lane information from the high-precision map and lane line information from the vehicle camera, perform multi-sensor data fusion and correlation and tracking, and give a lane structure result that combines the lane lines from the high-precision map and the perceived lane lines; if either the perceived lane line or the lane line from the high-precision map is lost, no data correlation and tracking will be performed, and the lane line result from a single sensor will be directly issued.

[0151] Step Two: Each frame of the camera image contains information about lane lines and obstacles. The results of lane line detection and obstacle detection in image perception can be reflected in the same frame, such as... Figure 4 As shown, the curve parameters of the perceived lane line in the image coordinate system and the coordinates of the four corner points of the obstacle's bounding box in the image coordinate system are obtained in the same frame.

[0152] Step 3: During vehicle operation, obstacles may be within a single lane or span two lanes (e.g., when a vehicle ahead changes lanes); the onboard camera may capture the complete outline of the obstacle or only a portion of it. Therefore, obstacle lane location can be broadly categorized into the following situations:

[0153] like Figure 4 As shown, Case 1: The obstacle is only in one lane, and the lane lines are not obstructed.

[0154] Calculate the lateral positional relationship between the bottom left and bottom right points of the obstacle envelope and the two lane lines (left_line[i] and right_line[i]) of each lane, and the ratio of the width (bottom_width) of the envelope:

[0155] left_over=(bottom_right-left_line[i]) / bottom_width

[0156] right_over=(bottom_left-right_line[i]) / bottom_width

[0157] If the obstacle is located in the lane the vehicle is currently traveling in, both its left_over and right_over values ​​are positive; if the obstacle is located in the right lane the vehicle is currently traveling in, its right_over value is negative; the same applies to the left lane. This allows us to determine which lane the obstacle is in; the results calculated on the 2D pixel plane are assigned a confidence level of 1.0.

[0158] like Figure 5 As shown, Case 2: The obstacle spans two lanes, and one lane line is obscured by the obstacle, making it an invalid lane line.

[0159] Calculate left_over and right_over in the manner described in case 1. If the left lane line of a certain lane is invalid, then only right_over can be used. If right_over is negative, the obstacle is located in the right lane of the current lane.

[0160] If right_over is a positive value, the obstacle is located in the current lane or the lane to the left of the current lane;

[0161] Only cases where right_over is negative, indicating that the obstacle is located in the right lane of the current lane, are considered and recorded in the detection results; the results calculated on the 2D pixel plane are assigned a confidence level of 1.0.

[0162] like Figure 6 As shown, Case 3: If the obstacle is incomplete in the image.

[0163] If the image perception result indicates that the obstacle is in a truncated state (visual perception will provide a flag indicating whether it is truncated), the lane it is in will still be calculated according to Case or Case method.

[0164] The lane localization result calculated on the 2D pixel plane for the truncated obstacle is assigned a confidence level of 0.2. Finally, it is compared with the result in 3D space, and the result with the higher confidence level is selected as the final result.

[0165] Step 4: In the current vehicle coordinate system, acquire the 3D envelope data, center point coordinates, and lane line information of the obstacle after multi-sensor data fusion. Based on the relative positions of the four corner points on the bottom surface of the 3D envelope with the lane lines, determine which lane the obstacle is in. If the obstacle spans two lanes, record both lanes in the result.

[0166] In 3D space, the accuracy of obstacle location information from multi-sensor data fusion and lane line information from perception and high-precision map fusion decreases with increasing distance. Therefore, in 3D space, the confidence level for lane line segmentation is set as: Conf = 0.2 + (1 - (x - x_min) / (x_max - x_min)) * 0.8, where x is the ordinate of the vehicle's center point, x_max is the ordinate of the furthest lane line position, and x_min is the ordinate of the closest lane line position.

[0167] Step 5: Compare the confidence levels of the lane segmentation results in 2D and 3D spaces, and take the result with the higher confidence level as the final result.

[0168] This application also provides an obstacle sensing device 700 for autonomous vehicles, such as... Figure 7 The diagram provided illustrates the structure of an obstacle sensing device 700 for autonomous vehicles according to an embodiment of this application. The obstacle sensing device 700 for autonomous vehicles includes at least: a 2D pixel planar positioning module 710, a 3D spatial positioning module 720, and a determination module 730, wherein:

[0169] In one embodiment of this application, the 2D pixel plane positioning module 710 is specifically used to: determine the position of the obstacle in the image coordinate system based on the camera perception results on the autonomous vehicle, and obtain the 2D lane positioning result.

[0170] To better determine the lane where the obstacle is located, lane information from the high-precision map data of the current driving road and lane line information from the camera on the autonomous vehicle can be obtained in advance. The two are then fused and correlated through multi-sensor data to provide a lane structure result that combines the lane lines from the high-precision map and the perceived lane lines. If either the perceived lane line or the high-precision map lane line is lost, no target data correlation and tracking is performed, and the lane line result from a single sensor can be directly issued.

[0171] Autonomous vehicles typically have multiple types of sensors, including LiDAR, millimeter-wave radar, and vision sensors, using these multiple types of sensors to achieve multi-sensor fusion perception results.

[0172] For visual sensors, multiple cameras may be installed on the autonomous vehicle. The specific installation location of the cameras is not specifically limited in the embodiments of this application, and those skilled in the art can choose according to the actual situation. For the camera perception results on the autonomous vehicle (including at least one camera), the position of the obstacle in the image coordinate system is determined in the 2D image pixels, which is used as the 2D lane positioning result.

[0173] It is important to note that at this point, it is not necessary to determine whether there is any occlusion in the perception result; it is only necessary to calculate the confidence level of the perception result later.

[0174] Furthermore, since each frame of the camera image contains information about lane lines and obstacles (images without this information are not considered results), the results of lane line detection and obstacle detection in image perception can both be reflected in the same frame, such as... Figure 4 As shown, the curve parameters of the perceived lane lines in the image coordinate system (e.g., lane lines, traffic signs) and the coordinates of the four corner points of the target obstacle's bounding box in the image coordinate system can be obtained in the same frame.

[0175] In one embodiment of this application, the 3D spatial positioning module 720 is specifically used to: determine the position of the obstacle in the vehicle coordinate system based on the multi-sensor perception fusion result on the autonomous vehicle, and obtain the 3D lane positioning result.

[0176] As mentioned earlier, autonomous vehicles include various types of sensors, and the position of the obstacle in the vehicle coordinate system can be determined based on the multi-sensor perception fusion results, thus obtaining the 3D lane positioning result in a three-dimensional scene.

[0177] Furthermore, in the current vehicle coordinate system, the 3D envelope data, center point coordinates, and lane line information of the obstacle are acquired after data fusion from multiple sensors (which could be cameras, LiDAR, or even more sensors). Based on the relative positions of the four corner points on the bottom surface of the 3D envelope with the lane lines, the lane in which the obstacle is located is determined. In other words, the lane in which the target obstacle is located can be determined through the multi-sensor perception fusion result. Since high-precision map data is available, the lane in which the target obstacle is located can be determined based on the fused perception result.

[0178] For example, either visual SLAM or laser SLAM can be used to determine which specific lane an obstacle is in.

[0179] It is important to note that in 3D space, the accuracy of both obstacle location information obtained from multi-sensor data fusion and lane line information obtained from the fusion of perception and high-precision maps decreases as the distance increases.

[0180] In one embodiment of this application, the determining module 730 is specifically used to: determine the lane where the obstacle is located based on the 2D lane positioning result and the 3D lane positioning result.

[0181] Based on the 2D lane positioning results and the 3D lane positioning results obtained above, the confidence levels of the lane division results in 2D and 3D spaces are compared, and the result with the higher confidence level is taken as the final result.

[0182] If only the lane line results perceived by the camera are used, the effective distance is usually within 80-100 meters. When encountering curves, the effective distance of lane line perception will be even shorter. 3D lane positioning results can solve the problem that when the longitudinal position of an obstacle exceeds the effective range of the perceived lane line, there are no lane lines around, and it is impossible to determine which lane the obstacle is in based on its relative position.

[0183] It is understood that the obstacle perception device for autonomous vehicles described above can implement each step of the obstacle perception method for autonomous vehicles provided in the foregoing embodiments. The relevant explanations of the obstacle perception method for autonomous vehicles are applicable to the obstacle perception device for autonomous vehicles, and will not be repeated here.

[0184] Figure 8 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Please refer to it. Figure 8At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.

[0185] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 8 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0186] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0187] The processor reads the corresponding computer program from non-volatile memory into main memory and then runs it, forming the obstacle perception device for autonomous vehicles at the logical level. The processor executes the program stored in memory and specifically performs the following operations:

[0188] Based on the camera perception results on the autonomous vehicle, the position of the obstacle in the image coordinate system is determined, and the 2D lane localization result is obtained.

[0189] Based on the multi-sensor perception fusion results on the autonomous vehicle, the position of the obstacle in the vehicle coordinate system is determined, and the 3D lane positioning result is obtained.

[0190] Based on the 2D lane positioning results and the 3D lane positioning results, the lane where the obstacle is located is determined.

[0191] The above is as stated in this application. Figure 1The method for obstacle perception in an autonomous vehicle disclosed in the illustrated embodiments can be applied to or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0192] The electronic device can also perform Figure 1 A method for using an obstacle sensing device in an autonomous vehicle is described, and the method for implementing the obstacle sensing device in an autonomous vehicle is described. Figure 1 The functions of the embodiments shown are not described in detail here.

[0193] This application also proposes a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by an electronic device including multiple applications, enable the electronic device to perform... Figure 1 The method executed by the obstacle perception device for an autonomous vehicle in the illustrated embodiment is specifically used to perform:

[0194] Based on the camera perception results on the autonomous vehicle, the position of the obstacle in the image coordinate system is determined, and the 2D lane localization result is obtained.

[0195] Based on the multi-sensor perception fusion results on the autonomous vehicle, the position of the obstacle in the vehicle coordinate system is determined, and a 3D lane positioning result is obtained.

[0196] Based on the 2D lane positioning results and the 3D lane positioning results, the lane where the obstacle is located is determined.

[0197] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0198] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0199] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0200] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0201] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0202] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0203] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0204] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0205] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0206] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. An obstacle perception method for autonomous vehicles, wherein, The method includes: Based on the camera perception results on the autonomous vehicle, the position of the obstacle in the image coordinate system is determined to obtain the 2D lane localization result; Based on the multi-sensor perception fusion results on the autonomous vehicle, the position of the obstacle in the vehicle coordinate system is determined, and a 3D lane positioning result is obtained. Based on the 2D lane positioning results and the 3D lane positioning results, the lane in which the obstacle is located is determined; The step of determining the position of the obstacle in the vehicle coordinate system based on the multi-sensor perception fusion results on the autonomous vehicle to obtain 3D lane positioning results includes: In the vehicle coordinate system of an autonomous vehicle, the 3D envelope data, center point coordinates, and lane line information of the obstacle are obtained after the fusion of multi-sensor data. Based on the relative positions of the four corner points on the bottom surface of the 3D envelope and the lane lines, determine which lane the obstacle is in; If the obstacle is in one lane, then that lane is recorded; if the obstacle spans two lanes, then both lanes are recorded. Based on the lane recording results, the 3D lane positioning results calculated in 3D space are assigned a third confidence level, which is calculated as follows: Conf = 0.2 + (1 - (x - x_min) / (x_max - x_min))*0.8, where X is the ordinate of the vehicle's center point, x_max is the ordinate of the furthest position of the lane line, and x_min is the ordinate of the closest position of the lane line; Determining the lane where the obstacle is located based on the 2D lane positioning result and the 3D lane positioning result includes: The confidence levels of the 2D lane positioning results and the 3D lane positioning results are compared, and the result with the higher confidence level is taken as the final result to determine the lane where the obstacle is located.

2. The method as described in claim 1, wherein, The method further includes: Pre-acquire lane information from high-precision map data of the current driving segment of the autonomous vehicle; The lane line information from the camera perception results on the autonomous vehicle is fused with the lane information in the high-precision map data and then correlated and tracked to obtain lane structure information based on the lane lines in the high-precision map and the lane lines perceived by the camera. The lane structure information includes at least lane ID and lane line ID.

3. The method as described in claim 2, wherein, The step of determining the position of the obstacle in the image coordinate system based on the camera perception results on the autonomous vehicle to obtain the 2D lane positioning result includes: Based on the camera perception results on the autonomous vehicle, obtain the curve parameters of the perceived lane line in the image coordinate system and the coordinates of the four corner points of the obstacle's envelope in the image coordinate system on the same image frame; Based on the curve parameters and the coordinates of the four corner points of the envelope, the accurate position of the obstacle in the image coordinate system is determined, and the 2D lane positioning result is obtained.

4. The method as described in claim 3, wherein, The step of determining the accurate position of the obstacle in the image coordinate system based on the curve parameters and the coordinates of the four corner points of the envelope to obtain the 2D lane positioning result includes: If the lane lines are not obscured by the obstacle, all lanes in the lane structure are traversed according to the lane structure information, and the proportional relationship between the obstacle and the left and right lane lines of one of the lanes is calculated. The lane in which the obstacle is located is determined based on the proportional relationship between the obstacle and the left and right lane lines of one of the lanes. If both the lower left and lower right points of the obstacle are inside a lane, then the obstacle is in the current lane. If the obstacle spans two lanes, the proportion of the obstacle's width in the two lanes is calculated, and it is determined whether the obstacle is in the current lane or spans two lanes.

5. The method as described in claim 4, wherein, The step of determining the accurate position of the obstacle in the image coordinate system based on the curve parameters and the coordinates of the four corner points of the envelope to obtain the 2D lane positioning result further includes: Calculate the horizontal positional relationship between the bottom left and bottom right points of the obstacle's bounding box and the left and right lane lines of each lane, and the ratio of the bounding box width to bottom width. The left and right lane lines are the horizontal positions of the lane lines in the image coordinate system. If the obstacle is located in the lane where the current autonomous vehicle is traveling, then both left_over and right_over in the preset proportional relationship are positive values; If the obstacle is located in the right lane of the vehicle currently traveling, then the right_over value is negative; If the obstacle is located in the left lane of the vehicle currently traveling, then right_over is a positive value; The obstacle is located in which lane ID, and the 2D lane positioning result calculated on the 2D pixel plane is assigned a first confidence level.

6. The method of claim 5, wherein, The step of determining the accurate position of the obstacle in the image coordinate system based on the curve parameters and the coordinates of the four corner points of the envelope to obtain the 2D lane positioning result further includes: When the obstacle spans two lanes, and one lane line is obscured by the obstacle and is therefore an invalid lane line: If the left lane line of a lane is invalid, then use right_over; If right_over is negative, the obstacle is located in the right lane of the current lane; If right_over is a positive value, then the obstacle is located in the current lane or the left lane of the current lane; The case where the right_over is negative and the obstacle is located in the right lane ID of the current lane is recorded as the detection result, and the 2D lane positioning result calculated on the 2D pixel plane is assigned a first confidence level.

7. The method of claim 4, wherein, The method further includes: When the obstacle is incomplete in the image: If the obstacle is determined to be in a truncated state based on the truncated flag in the image perception results of the autonomous vehicle, then the lane in which the truncated obstacle is located is calculated based on the obtained truncated obstacle. Based on the 2D lane positioning result calculated on the 2D pixel plane of the truncated obstacle, a second confidence level is assigned, which is less than the first confidence level.

8. The method according to any one of claims 1 to 7, wherein, Determining the lane where the obstacle is located based on the 2D lane positioning result and the 3D lane positioning result includes: The confidence levels of the 2D lane positioning results and the 3D lane positioning results are compared, and the result with the higher confidence level is taken as the final result to determine the lane where the obstacle is located.

9. An obstacle sensing device for an autonomous vehicle, wherein, The device includes: The 2D pixel plane positioning module is used to determine the position of obstacles in the image coordinate system based on the camera perception results on the autonomous vehicle, and obtain 2D lane positioning results. The 3D spatial positioning module is used to determine the position of the obstacle in the vehicle coordinate system based on the multi-sensor perception fusion results on the autonomous vehicle, and obtain the 3D lane positioning result. The determination module is used to determine the lane where the obstacle is located based on the 2D lane positioning results and the 3D lane positioning results; The step of determining the position of the obstacle in the vehicle coordinate system based on the multi-sensor perception fusion results on the autonomous vehicle to obtain 3D lane positioning results includes: In the vehicle coordinate system of an autonomous vehicle, the 3D envelope data, center point coordinates, and lane line information of the obstacle are obtained after the fusion of multi-sensor data. Based on the relative positions of the four corner points on the bottom surface of the 3D envelope and the lane lines, determine which lane the obstacle is in; If the obstacle is in one lane, then that lane is recorded; if the obstacle spans two lanes, then both lanes are recorded. Based on the lane recording results, the 3D lane positioning results calculated in 3D space are assigned a third confidence level, which is calculated as follows: Conf = 0.2 + (1 - (x - x_min) / (x_max - x_min))*0.8, where X is the ordinate of the vehicle's center point, x_max is the ordinate of the furthest position of the lane line, and x_min is the ordinate of the closest position of the lane line; Determining the lane where the obstacle is located based on the 2D lane positioning result and the 3D lane positioning result includes: The confidence levels of the 2D lane positioning results and the 3D lane positioning results are compared, and the result with the higher confidence level is taken as the final result to determine the lane where the obstacle is located.

10. An electronic device, comprising: processor; as well as A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the method of any one of claims 1 to 8.

11. A computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of applications, cause the electronic device to perform the method of any one of claims 1 to 8.

Images