Road element detection method, apparatus, and vehicle
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- YINWANG INTELLIGENT TECHNOLOGIES CO LTD
- Filing Date
- 2024-10-30
- Publication Date
- 2026-07-03
Smart Images
Figure CN122341997A_ABST
Abstract
Description
Road element detection methods, devices and vehicles Technical Field
[0001] This application relates to the field of intelligent driving, and more specifically, to a method, apparatus, and vehicle for detecting road elements. Background Technology
[0002] With the rapid development of the automotive industry, many driver assistance and autonomous driving technologies have emerged, which can reduce driving stress and improve safety and traffic efficiency. Currently, most autonomous driving technologies rely on high-precision maps for navigation. However, high-precision maps have drawbacks such as high collection and production costs, long processing times, insufficient coverage, and difficulty in ensuring data freshness, making it difficult to promote autonomous driving technologies that rely on high-precision maps nationwide or globally.
[0003] Intelligent vehicles use sensors such as cameras and LiDAR to detect static elements (such as lanes, roads, traffic lights, and signs) within a certain range around the vehicle in real time. This allows them to model the surrounding environment, enabling vehicles to navigate smoothly in complex and ever-changing traffic conditions without relying on high-precision maps. However, the current sensor range of vehicles is limited, resulting in some road elements not being accurately identified, which significantly impacts driving safety and traffic efficiency.
[0004] Therefore, a road element detection scheme that can improve the completeness and accuracy of road element recognition urgently needs to be developed.
[0005] Summary of the Invention
[0006] This application provides a road element detection method, apparatus, and vehicle, which helps to improve the integrity and accuracy of identified road boundaries, thereby improving vehicle driving safety and traffic efficiency.
[0007] Firstly, a road element detection method is provided, which can be performed by a vehicle, for example, by the vehicle's computing platform, or by a chip or circuitry used in the vehicle.
[0008] The method includes: acquiring environmental perception information, which at least indicates a boundary of a first road where the vehicle is located, and a first intersection connecting to the first road; inferring a first boundary and a second boundary of the first road based on a first model according to the environmental perception information and the type of the first intersection; wherein the area between the first boundary and the second boundary belongs to a one-way road.
[0009] In some implementations, environmental perception information may include at least one of the following: images acquired by visual sensors such as cameras, or point cloud data acquired by sensors such as lidar.
[0010] In some implementations, the types of the first intersection include: n-way intersections (such as three-way intersections, crossroads, etc.), merging intersections (such as ramps where auxiliary roads merge into main roads), split intersections (such as ramps where vehicles exit the main road), etc.
[0011] In some implementations, the first road is a one-way road, and the first and second boundaries can be the left and right boundaries of the one-way road relative to the vehicle, respectively. This one-way road may include one lane or multiple lanes. In other implementations, the first road is a two-way road, where the first and second boundaries can be the left and right boundaries of the one-way road relative to the vehicle.
[0012] In some implementations, the area between the first boundary and the second boundary is the drivable area for vehicles. The aforementioned drivable area can be understood as: the area where vehicles are allowed to drive under traffic rules; or, the area where vehicles are allowed to drive under safety conditions.
[0013] It is understandable that the width of a road generally changes near an intersection. Therefore, even when the environmental perception information only indicates one boundary of the first road, the paired boundaries of the first road can be inferred based on the topological relationship between the first intersection and the first road, as well as the type of the first intersection. This can improve the scalability of the vehicle's sensors in terms of perception accuracy and range. In other words, it can infer road elements that the sensors have not perceived, thereby improving the completeness and accuracy of road boundary recognition. Generally speaking, for scenarios such as narrow / continuous roads merging and merging, and parallel multi-intersections, it is impossible to accurately determine the changes in lane line positions at road forks and intersections, leading to unexpected vehicle deviations. For example, a vehicle is traveling on the main road and needs to exit the main road via a ramp at the intersection ahead. However, because the lane line recognition results do not indicate or cannot accurately show the positions of road forks and / or intersections, the vehicle misses the intersection and deviates. Or, when the vehicle is close to the intersection, the intersection is recognized, and the vehicle may suddenly turn, introducing unsafe factors. In the aforementioned scenario, where road elements such as lane lines cannot be reliably and accurately identified, vehicles can travel based on paired road boundaries, reducing the likelihood of vehicle deviation and thus improving driving safety and traffic efficiency.
[0014] In conjunction with the first aspect, in some implementations of the first aspect, reasoning about the first boundary and the second boundary of the first road includes: reasoning about the first boundary based on the boundary of the first road indicated by environmental perception information; determining the width of the first road based on the type of the first intersection; and reasoning about the second boundary based on the first boundary and the width of the first road.
[0015] In the above technical solution, inferring the width of the road based on the type of the first intersection helps to improve the accuracy of the inferred first road boundary.
[0016] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: obtaining lane type information, the lane type information indicating the type of at least one lane in the first road; and inferring the first boundary and the second boundary of the first road, including: inferring the first boundary and the second boundary based on environmental perception information, lane type information, and the type of the first intersection.
[0017] In the above technical solution, the boundary of the area for vehicle travel (i.e., the driving area) in the first road is determined based on the lane type. This makes it easier for vehicles to avoid unconventional lanes when planning their driving routes, thereby reducing the probability of vehicle violations and improving vehicle driving safety.
[0018] In conjunction with the first aspect, in some implementations of the first aspect, the first road is a one-way road, the first road includes a first lane, and when the lane type information indicates that the first lane is an unconventional lane, the area between the first boundary and the second boundary does not include the first lane; or, when the lane type information indicates that the first lane is a regular lane, the area between the first boundary and the second boundary includes the first lane.
[0019] Specifically, lane type information can indicate whether there are unconventional lanes in the first road, and the location of unconventional lanes in the first road, such as the start and end positions of unconventional lanes in the first road. Unconventional lanes can be understood as lanes that are normally not permitted for vehicles to travel in.
[0020] Since certain types of vehicles are not permitted to travel in unconventional lanes, when a vehicle belongs to one of these types, determining the boundaries based on lane type can exclude such lanes from the vehicle's driving area, reducing the likelihood of traffic violations and preventing unsafe factors caused by vehicles mistakenly entering unconventional lanes.
[0021] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: inferring the position of at least one lane line in the first road and the boundary of the first intersection based on environmental perception information; inferring the first boundary and the second boundary of the first road, including: inferring the first boundary and the second boundary based on the position of at least one lane line, the boundary of the first intersection, and the type of the first intersection.
[0022] In the above technical solution, the predicted positions of the first and second boundaries are corrected based on the position of the lane lines in the first road, the boundary of the first intersection, and the degree of fit between the first and second boundaries of the first road, which helps to improve the accuracy of road boundary identification.
[0023] In conjunction with the first aspect, in some implementations of the first aspect, the first road is connected to the first intersection boundary of the first intersection, the environmental perception information also indicates at least one boundary including the second road, the second road is connected to the second intersection boundary of the first intersection, and the second road is the road that the vehicle travels to its destination, the method further includes: inferring the boundary of the second road based on the environmental perception information, the type of the first intersection, the first boundary and the second boundary.
[0024] It is understandable that, given the boundaries of the second road and the first road, the topological relationship between the first road, the first intersection, and the second road can be deduced.
[0025] In the above technical solution, the boundary of the target road (such as the second road) can be deduced based on the boundary of the road the vehicle is currently on. This helps improve the timeliness of detecting road boundaries and road topology changes, reducing problems such as unexpected vehicle steering and yaw caused by untimely road boundary detection. Furthermore, even when road elements such as lane lines and intersections have poor alignment with road boundaries, the vehicle can still travel based on the topological relationship between the first road, the first intersection, and the second road, as well as the deduced boundaries of the first and second roads, thus improving traffic efficiency.
[0026] In conjunction with the first aspect, in some implementations of the first aspect, the first model is trained based on multiple sets of road boundary ground values. Each set of road boundary ground values indicates the actual location of the road boundary, and each set of road boundary ground values includes the boundaries on both sides of the road.
[0027] In the above technical solution, training a neural network model for detecting road elements by using paired road boundaries helps improve the neural network model's ability to recognize paired road boundaries, thereby improving the self-consistency between the road boundaries, lane lines, and intersection boundaries output by the neural network model.
[0028] In conjunction with the first aspect, in some implementations of the first aspect, the true value of the road boundary is obtained by dividing the reference road based on intersections and / or unconventional lanes, and the reference road indicates the actual boundary position of the road.
[0029] In the above technical solution, after training the neural network model based on the ground truth of road boundaries obtained from intersections and unconventional lanes, the detection rate and recognition accuracy of the neural network model for areas where the road topology changes will be improved, thereby reducing the instability of road topology output and reducing problems such as unexpected vehicle steering and yaw caused by unstable or untimely road topology detection.
[0030] In conjunction with the first aspect, in some implementations of the first aspect, the first model can be an end-to-end neural network model.
[0031] Secondly, a control method is provided that can be executed by a vehicle, for example, by the vehicle's computing platform, or by a chip or circuitry for the vehicle.
[0032] The method includes controlling the vehicle to travel in the area between a first boundary and a second boundary in any possible implementation of the first aspect.
[0033] In the above technical solution, controlling vehicle driving based on the road boundary detected by the first aspect helps to reduce the instability of road topology output, reduce problems such as unexpected vehicle turning and yaw caused by unstable or untimely road topology detection, thereby improving vehicle driving safety.
[0034] Thirdly, a road element detection device is provided, the device comprising an acquisition unit and a processing unit, wherein the acquisition unit is configured to: acquire environmental perception information, the environmental perception information indicating at least one boundary of a first road where the vehicle is located, and a first intersection connecting to the first road; the processing unit is configured to: infer a first boundary and a second boundary of the first road based on a first model according to the environmental perception information and the type of the first intersection; wherein the area between the first boundary and the second boundary belongs to a one-way road.
[0035] In conjunction with the third aspect, in some implementations of the third aspect, the processing unit is used to: infer a first boundary based on the boundary of the first road indicated by environmental perception information; determine the width of the first road based on the type of the first intersection; and infer a second boundary based on the first boundary and the width of the first road.
[0036] In conjunction with the third aspect, in some implementations of the third aspect, the acquisition unit is further configured to: acquire lane type information, the lane type information indicating the type of at least one lane in the first road; the processing unit is further configured to: infer the first boundary and the second boundary based on the environmental perception information, the lane type information, and the type of the first intersection.
[0037] In conjunction with the third aspect, in some implementations of the third aspect, the first road is a one-way road, the first road includes a first lane, and when the lane type information indicates that the first lane is an unconventional lane, the area between the first boundary and the second boundary does not include the first lane; or, when the lane type information indicates that the first lane is a regular lane, the area between the first boundary and the second boundary includes the first lane.
[0038] In conjunction with the third aspect, in some implementations of the third aspect, the processing unit is also used to: infer the position of at least one lane line in the first road and the boundary of the first intersection based on environmental perception information; and infer the first boundary and the second boundary based on the position of at least one lane line, the boundary of the first intersection, and the type of the first intersection.
[0039] In conjunction with the third aspect, in some implementations of the third aspect, the first road is connected to the first intersection boundary of the first intersection, the environmental perception information also indicates at least one boundary including the second road, the second road is connected to the second intersection boundary of the first intersection, and the second road is the road that the vehicle travels to its destination, and the processing unit is also used to: infer the boundary of the second road based on the environmental perception information, the type of the first intersection, the first boundary and the second boundary.
[0040] In conjunction with the third aspect, in some implementations of the third aspect, the processing unit is used to: input environmental perception information into the first model to obtain the first boundary and the second boundary; wherein the first model is trained based on multiple sets of road boundary ground values, each set of road boundary ground values indicates the actual location of the road boundary, and each set of road boundary ground values includes the boundaries on both sides of the road.
[0041] In conjunction with the third aspect, in some implementations of the third aspect, the true value of the road boundary is obtained by dividing the reference road based on intersections and / or unconventional lanes, and the reference road indicates the actual boundary position of the road.
[0042] Fourthly, a control device is provided, the device including a processing unit for: controlling the position of a vehicle in a drivable area according to a first boundary and a second boundary in any possible implementation of the first aspect.
[0043] Fifthly, a road element detection device is provided, the device including a processor for executing a computer program stored in the memory, such that the device performs the method in any possible implementation of the first aspect described above.
[0044] In a sixth aspect, a control device is provided, the device including a processor for executing a computer program stored in the memory, such that the device performs the method in any possible implementation of the second aspect described above.
[0045] In conjunction with the fifth or sixth aspect, in some implementations of the fifth or sixth aspect, the device also includes a memory.
[0046] In a seventh aspect, a computer program product is provided, comprising: computer program code, which, when executed on a computer or processor, causes the computer or processor to perform the method in any possible implementation of the first or second aspect.
[0047] It should be noted that the above computer program code can be stored in whole or in part on a storage medium, which can be packaged together with the processor or packaged separately from the processor.
[0048] Eighthly, a computer-readable storage medium is provided, the computer-readable medium storing instructions that, when executed by a processor, cause the processor to implement the method in any possible implementation of the first or second aspect.
[0049] Ninthly, a chip is provided, the chip including circuitry for performing the methods in any possible implementation of the first or second aspect described above.
[0050] In a tenth aspect, a vehicle is provided that includes means as in any of the possible implementations of the third to sixth aspects, or the vehicle includes computer-readable storage as in any of the possible implementations of the eighth aspect, or the vehicle includes a chip as in any of the possible implementations of the ninth aspect, or the vehicle is loaded with computer program code as in any of the possible implementations of the seventh aspect.
[0051] In conjunction with aspect ten, in some implementations of aspect ten, the vehicle is a vehicle in a broad sense, such as a means of transportation (e.g., commercial vehicles, passenger cars, motorcycles, flying cars, trains, etc.), industrial vehicles (e.g., forklifts, trailers, tractors, etc.), engineering vehicles (e.g., excavators, bulldozers, cranes, etc.), agricultural equipment (e.g., lawnmowers, harvesters, etc.), amusement equipment, toy vehicles, etc. In practical implementation, the vehicle can also be a road vehicle, a water vehicle, an air vehicle, industrial equipment, agricultural equipment, or other intelligent driving equipment such as entertainment equipment.
[0052] For the beneficial effects not described in detail in aspects two through ten, please refer to the description in aspect one, which will not be repeated here. Attached Figure Description
[0053] Figure 1 is a functional schematic block diagram of the vehicle provided in an embodiment of this application;
[0054] Figure 2 is a schematic block diagram of the road element detection system architecture provided in an embodiment of this application;
[0055] Figure 3 is a schematic flowchart of a training method for a road element detection model provided in an embodiment of this application;
[0056] Figure 4 is a schematic diagram of road elements involved in an embodiment of this application;
[0057] Figure 5 is another schematic diagram of road elements provided in an embodiment of this application;
[0058] Figure 6 is another schematic diagram of road elements provided in an embodiment of this application;
[0059] Figure 7 is a schematic diagram of the neural network model provided in an embodiment of this application;
[0060] Figure 8 is a schematic flowchart of the road element detection method provided in an embodiment of this application;
[0061] Figure 9 is a schematic diagram of the relative positional relationships between road elements provided in an embodiment of this application;
[0062] Figure 10 is a schematic diagram of the application scenario provided in the embodiments of this application;
[0063] Figure 11 is a schematic block diagram of the road element detection device provided in an embodiment of this application;
[0064] Figure 12 is another schematic block diagram of the road element detection device provided in the embodiments of this application. Detailed Implementation
[0065] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.
[0066] Figure 1 is a functional block diagram of a vehicle provided in an embodiment of this application. As shown in Figure 1, the vehicle 100 may include a perception system 120 and a computing platform 150. The perception system 120 may include several sensors for sensing information about the environment surrounding the vehicle 100. For example, the perception system 120 may include a positioning system, which may be a global navigation satellite system (GNSS), such as the global positioning system (GPS), BeiDou system, etc. Alternatively, the perception system 120 may also include one or more of the following: an inertial measurement unit (IMU), lidar, millimeter-wave radar, ultrasonic radar, and a camera device.
[0067] For example, the camera device for acquiring images of the exterior of the vehicle may include one or more of a front-view camera, a rear-view camera, a surround-view camera, and a side-view camera. The front-view camera may be mounted on the windshield. The rear-view camera may be mounted in the trunk. The side-view camera may be mounted below the rearview mirror. The surround-view camera includes four cameras mounted around the vehicle; the images acquired by the four cameras are stitched together to obtain a panoramic image of the vehicle's surroundings. In some implementations, the surround-view camera may overlap with the front-view, rear-view, and side-view cameras. For example, the camera positioned on the side of the vehicle in the surround-view camera may be a side-view camera, the camera positioned in front of the vehicle may be a front-view camera, and the camera positioned behind the vehicle may be a rear-view camera. Alternatively, the surround-view camera may be different from all three cameras: the front-view camera, the rear-view camera, and the side-view camera.
[0068] Some or all of the functions of vehicle 100 can be controlled by computing platform 150. Computing platform 150 may include processors 151 to 15n. A processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction read and execute capabilities, such as a central processing unit (CPU), microprocessor, graphics processing unit (GPU) (which can be understood as a type of microprocessor), or digital signal processor (DSP). In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. These logical relationships are fixed or reconfigurable. For example, the processor may be a hardware circuit implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as a field-programmable gate array (FPGA). In reconfigurable hardware circuits, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the process of the processor loading instructions to implement some or all of the functions of the aforementioned units. Furthermore, the processor can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), tensor processing unit (TPU), deep learning processing unit (DPU), etc. In addition, the computing platform 150 may also include memory for storing instructions. Some or all of the processors 151 to 15n can call instructions from the memory to implement corresponding functions.
[0069] In this application, the computing platform 150 can predict the intersection boundary and road boundary of the road where the vehicle is located based on the environmental information collected by the perception system 120. The roles of the perception system 120 and the computing platform 150 in road boundary detection are explained in detail below with reference to Figure 2.
[0070] Figure 2 shows a schematic block diagram of the road boundary detection system architecture provided in an embodiment of this application. The system includes a perception module 210 and a detection module 220. In some implementations, the system may also include a control module 230. Exemplarily, the perception module 210 may include one or more sensors from the perception system 120 shown in Figure 1; the detection module 220 may include one or more processors from the computing platform 150 shown in Figure 1, or the detection module 220 may include one or more processors from a cloud server communicating with vehicles; the control module 230 may include one or more processors from the computing platform 150 shown in Figure 1. The functions of each module are as described in items (a) to (iii) below.
[0071] (i) The perception module 210 is used to collect environmental information around the vehicle. This environmental information can indicate the road structure around the vehicle, and the form of the environmental information can be images, laser point clouds, etc. The perception module 210 can send the collected environmental information to the detection module 220.
[0072] (ii) The detection module 220 is used to determine the road boundary of the road where the vehicle is located based on the environmental information collected by the perception module 210. More specifically, the detection module 220 may include a training ground truth construction module 221, a model training module 222, and an element extraction module 223. In some implementations, the detection module 220 may also include a topology construction module 224. The training ground truth construction module 221 is used to obtain the road boundary ground truth based on the labeled road data. This road boundary ground truth indicates the true boundary of the road, specifically a boundary pair consisting of the left and right boundaries. In some implementations, the training ground truth construction module 221 may also obtain the intersection boundary ground truth and lane line ground truth based on the labeled road data. The intersection boundary ground truth indicates the position where the road connects to the intersection; for example, the intersection boundary can be formed by the stop line at the intersection. The element extraction module 223 is used to predict the road boundary of the road where the vehicle is located based on the environment collected by the perception module 210. The model training module 222 can use the ground truth of the road boundary to supervise the training of the element extraction module 223, so as to improve the accuracy of the road elements predicted by the element extraction module 223. The topology construction module 224 can construct the topological relationship between roads and intersections based on the predicted road boundaries and intersection boundaries.
[0073] (iii) The planning and control module 230 can plan a driving path for the vehicle based on the road elements predicted by the detection module 220, and control the vehicle to drive along the planned driving path.
[0074] It should be understood that the above modules are only an example, and in actual applications, these modules may be added or removed as needed. For example, in the system architecture shown in Figure 2, the sensing module 210 and the detection module 220 can be merged into one module; or, the detection module 220 and the control module 230 can be merged into one module.
[0075] The above describes the road element detection system architecture provided in the embodiments of this application. The following details the process of implementing the road element detection method provided in the embodiments of this application based on the system architecture shown in Figure 2.
[0076] Figure 3 shows a schematic flowchart of a training method for a road element detection model. This method can be executed by vehicle 100 or by a cloud server communicating with vehicle 100. Exemplarily, the method can be executed by the detection module 220 shown in Figure 2. More specifically, steps S301 and S302 can be executed by the training ground truth construction module 221, and step S303 can be executed by the model training module 222.
[0077] S301, acquire road data, which includes road boundary data and intersection boundary data.
[0078] In some implementations, road data can be data in the BEV coordinate system. This road data can include manually labeled road boundary data and intersection boundary data. Road boundary data indicates the location of the left and right boundaries of at least one road, and intersection boundary data indicates the location of intersection boundaries. Both road boundary data and intersection boundary data also indicate the topological relationship between the intersection boundary and at least one road. In practical implementations, road data can also indicate information such as lane line positions and lane types within the road.
[0079] S302, determine the true value of the road boundary based on road data.
[0080] For example, determining the true value of the road boundary based on road data may include: selecting road data (hereinafter referred to as selected road data) from the road data within a first range from the vehicle based on the vehicle's location, and then determining the true value of the road boundary based on the selected road data.
[0081] In one example, the first range can be the area within a circle centered on the vehicle's center and with a radius of a preset length. In another example, the first range can also be the area within a semicircle centered on the vehicle's center and with a radius of a preset length, wherein the straight side of the semicircle coincides with the Y-axis of the vehicle's overall coordinate system, and the center of the semicircle coincides with the origin O of the overall coordinate system. When the vehicle is traveling forward (i.e., towards the front of the vehicle), the arc of the semicircle is located on the positive half-axis of the X-axis; when the vehicle is traveling backward (i.e., towards the rear of the vehicle), the arc of the semicircle is located on the negative half-axis of the X-axis. It should be noted that the aforementioned preset length can be less than the detection distance of the vehicle's sensing system; that is, the first range is less than the range that the vehicle's sensing system can perceive. In yet another example, the first range can be the range that the vehicle's sensing system can perceive.
[0082] In some implementations, one or more reference roads are generated based on road boundary data and successor / successor relationships in the selected road data. Each reference road indicates the direction of a one-way road within a first range, and when the selected road data includes intersection boundary data, at least one of the reference roads passes through an intersection. The successor / successor relationship refers to the connection relationship between roads. For example, for multiple sub-roads in a one-way road, if part or all of sub-road 1 is located before sub-road 2, then sub-road 1 can be considered the predecessor road of sub-road 2, and sub-road 2 can be considered the successor road of sub-road 1.
[0083] For example, the road boundary data indicates the road boundary corresponding to each segment of a multi-segment road. If two segments of a multi-segment road have a successor-successor relationship, the two segments can be connected to obtain a part of a reference road. Similarly, one or more reference roads can be obtained based on the road data.
[0084] Furthermore, the reference road can be segmented based on one or more of the following factors in the selected road data: intersection location, intersection type, or lane type and lane boundaries. For example, when the reference road passes through an intersection, it is segmented into at least two sub-roads, including the portion before entering the intersection and the portion after exiting the intersection; when one end of the reference road is an intersection, the intersection portion is removed from the reference road. The intersection type can include one or more of the following: crossroads, merging intersections, and split intersections. As another example, when the lane type indicates the presence of non-motorized vehicle lanes, dedicated lanes, or other unconventional lanes, the reference road can also be segmented based on the location of these unconventional lanes.
[0085] Among them, unconventional lanes are lanes that are used by most motor vehicles, while regular lanes are used by specific vehicles, such as non-motorized vehicles, buses, or vehicles in emergency situations.
[0086] The segmented reference road is sampled and interpolated to obtain the ground truth value of the road boundary of at least one segment of one-way road, which includes the left and right boundaries of the one-way road.
[0087] In some implementations, the obtained road boundary ground truth values can be post-processed to delete invalid road boundary ground truth values. Alternatively, inconsistent boundary smoothing and breakpoint continuation can be applied to the road boundary ground truth values. Invalid road boundary ground truth values can include: road boundary ground truth values with only one side boundary, road boundary ground truth values where the left and right sides intersect, etc.
[0088] S303 uses the ground truth of road boundaries as a supervision signal for model training to train the neural network model used to extract road elements.
[0089] In some implementations, the true values of multiple road boundaries are concatenated based on the successive relationships of the roads corresponding to the true values of multiple road boundaries, and then the concatenated true values of road boundaries are used for model training.
[0090] In some implementations, when the vehicle reaches location 1, it collects environmental perception information from sensors. This environmental perception information is then input into a neural network model used to extract road elements, thereby obtaining a predicted value of the road boundary around the vehicle's location. This predicted value is a static prediction. For example, the sensors may include cameras, lidar, etc.
[0091] In some implementations, a neural network model can be trained based on the ground truth value of the road boundary corresponding to position 1, so that the difference between the predicted value of the road boundary corresponding to position 1 and the ground truth value of the road boundary corresponding to position 1 is as small as possible.
[0092] For example, the solid lines in Figure 4(a) represent examples of the ground truth values of road boundaries corresponding to multiple one-way roads. More specifically, boundaries 411 and 412 are specific examples of the ground truth values of road boundaries corresponding to a one-way road, and the dashed line 401 is an example of an intersection boundary. The solid lines in Figure 4(b) represent examples of the predicted values of road boundaries corresponding to roads in a certain area, and the dashed lines in Figure 4(b) represent examples of the ground truth values of road boundaries corresponding to roads in that area. Using the ground truth values of road boundaries as supervision signals for model training can minimize the difference between the predicted values and the ground truth values of road boundaries.
[0093] Figure 4 shows a schematic diagram of multiple pairs of road edge ground values obtained by segmenting the reference road according to the intersection location. In addition to Figure 4, Figures 5 and 6 also show schematic diagrams of road boundary ground values determined based on other intersections, lane locations, and types.
[0094] In one example, if the road within the first range is as shown in Figure 5(a), and the road includes boundary 501, boundary 502, boundary 503 and boundary 504, and the road includes a merging intersection, then by dividing the reference road corresponding to the road based on the merging intersection, three pairs of road boundary truth values as shown in Figure 5(b) can be obtained, including: ① 5-a1 and 5-a2, which indicate the actual positions of boundary 502 and boundary 501 respectively; ② 5-b1 and 5-b2, which indicate the actual positions of boundary 502 and boundary 503 respectively; ③ 5-c1 and 5-c2, which indicate the actual positions of boundary 504 and boundary 501 respectively.
[0095] In another example, if the road within the first range is as shown in Figure 6(a), the road includes boundaries 601, 602, 603, 604, 605, and 606. The direction of travel on the road between boundaries 601 and 603 is opposite to the direction of travel on the road between boundaries 604 and 606, and the lane between boundaries 601 and 602 is a bicycle lane, while the lane between boundaries 605 and 606 includes a dedicated bus lane (a). Then, the road can be segmented into corresponding reference roads based on lane type and location. And based on the location of the unconventional lane, the true values of the road boundaries of the road segment where the unconventional lane is located can be determined. Specifically, four pairs of true values of road boundaries can be obtained as shown in Figure 6(b), including: ① 6-a1 and 6-a2, which indicate the actual positions of boundary 606 and boundary 604 respectively; ② 6-b1 and 6-b2, which indicate the actual positions of boundary 605 and boundary 604 respectively; ③ 6-c1 and 6-c2, which indicate the actual positions of boundary 606 and boundary 604 respectively; ④ 6-d1 and 6-d2, which indicate the actual positions of boundary 603 and boundary 602 respectively.
[0096] It is understandable that if the aforementioned neural network model is trained based on the ground truth values of the road boundaries shown in Figure 6(b), the road boundaries obtained by the neural network model during actual deduction indicate the area where motor vehicles can travel, rather than the location of the road edge (or curb). In actual implementation, when there are unconventional lanes in the road, the obtained ground truth values of the road boundaries can also indicate the actual location of the road edge (or curb). If the aforementioned neural network model is trained based on such ground truth values of road boundaries, the road boundaries obtained by the neural network model during actual deduction indicate the actual boundaries of the road, that is, the area between a pair of road boundaries includes the area where motor vehicles can travel, as well as the unconventional lanes.
[0097] The specific implementation of obtaining the true value of the aforementioned road boundary can be found in the descriptions in S301 and S302, and will not be repeated here.
[0098] In some implementations, the neural network model involved in this embodiment can be as shown in Figure 7, including a backbone network, a projector, and a BEV backbone network. This neural network model can be included in the element extraction module 223 shown in Figure 2. For example, environmental perception information can include images collected by camera devices, point cloud data from LiDAR, etc. Taking four camera devices installed around the vehicle as an example, the environmental perception information can include four images collected by the four camera devices respectively. Processing the environmental perception information through the neural network model shown in Figure 7 to obtain the predicted value of the road boundary can include the following steps:
[0099] (1) After inputting the four images into the backbone network, four two-dimensional image features can be output, namely visual image features from four perspectives. Furthermore, the four images can be input into a depth decoder to obtain the depth information of the pixels in each image. For example, the backbone network can be a convolutional neural network (CNN) model, such as the visual geometry group (VGG), AlexNet, LeNet, etc.
[0100] (2) By inputting the four image features and the depth information of the corresponding pixels of each image into the projector, the features of the image coordinate system can be mapped to the BEV coordinate system. It is understood that each sensor in the vehicle has its own coordinate system, and their output data or perception results will eventually be aggregated into the vehicle's overall coordinate system for processing. The mapping relationship between the overall coordinate system and the image coordinate system can be determined based on the extrinsic and intrinsic parameter matrices of each camera device. For the scene corresponding to the image obtained from the surround-view camera device, the purpose of the projector is to determine the rasterized representation of the scene in the BEV coordinate system. For example, taking the projector as LSS(lift, splat, shoot), each of the four image features can be a two-dimensional image feature. Then, LSS first performs an outer product operation on the two-dimensional image features and the depth information of the corresponding pixels to obtain a three-dimensional image feature that can be processed by CNN. Then, the three-dimensional image features are summed and pooled (such as z-accumulation, flattening) to achieve dimensionality reduction. Finally, the dimensionality-reduced features are stitched together to obtain the BEV features. It should be noted that the above explanation uses LSS as the projector. In actual implementation, the projector can also be other algorithms, such as Cam2BEV, PyrOccNet, etc.
[0101] (3) By inputting BEV features and query features into the BEV backbone network, the predicted values of static road boundaries can be obtained. For example, the BEV backbone network can be a Transformer-type neural network such as Q-Former or vision-language pre-training (VLP) model.
[0102] For example, the ground truth of road boundaries is used as a supervision signal for model training to adjust the weights associated with each component in the backbone network, projector, and BEV backbone network, thereby improving the accuracy of road boundary predictions by the neural network.
[0103] In some implementations, the neural network model in this embodiment can also predict road elements such as intersection boundaries and lane line positions in the road surrounding the vehicle's location based on environmental information collected by sensors.
[0104] By using the training method described above for the road element detection model, training the neural network based on paired road boundary ground values can improve the consistency and accuracy of the road element detection results output by the neural network.
[0105] Figure 8 shows a schematic flowchart of the road element detection method provided in an embodiment of this application. The method can be performed by a vehicle 100, for example, by the element extraction module 223 in the detection module 220 shown in Figure 2.
[0106] S810, acquires environmental perception information, which indicates at least one boundary of the first road where the vehicle is located, and the first intersection connecting to the first road.
[0107] S820, based on environmental perception information and the type of the first intersection, infers the first boundary and the second boundary of the first road based on the first model.
[0108] The area between the first and second boundaries is a one-way road.
[0109] For example, the types of the first intersection include: n-way intersections (such as three-way intersections, crossroads, etc.), merging intersections (such as ramps where auxiliary roads merge into main roads), split intersections (such as ramps where vehicles exit the main road), etc.
[0110] In some implementations, S820 can be further refined as follows: inferring a first boundary based on the boundary of the first road indicated by the environmental perception information; determining the width of the first road based on the type of the first intersection; and inferring a second boundary based on the first boundary and the width of the first road.
[0111] For example, the widths of roads connected by different types of intersections may vary. For example, by training the aforementioned neural network model using ground truth values of road boundaries and the topological relationships between road boundaries and intersection boundaries, the correlation between road width and intersection type between pairs of road boundaries can be obtained. Then, when the environmental perception information obtains incomplete road information (e.g., only one road boundary), the road width can be inferred based on the type of intersection connected to the road. Furthermore, based on one boundary detected by the vehicle's perception system and the inferred road width, the other road boundary can be inferred.
[0112] In some implementations, the method further includes: obtaining lane type information, which indicates the type of at least one lane in the first road; S820 can be further refined to: inferring the first boundary and the second boundary based on environmental perception information, lane type information, and the type of the first intersection.
[0113] In some implementations, obtaining lane type information may include: determining lane type information based on environmental perception information, that is, identifying the type of each lane in the first road based on information perceived by vehicle sensors. Alternatively, obtaining lane type information may also include: receiving lane type information from a cloud server.
[0114] Specifically, lane type information can indicate whether there are unconventional lanes in the first road, and the location of unconventional lanes in the first road, such as the start and end positions of unconventional lanes in the first road. Unconventional lanes can be understood as lanes that are normally not permitted for vehicles to travel in.
[0115] In some implementations, the first road can be segmented based on lane type information, dividing it into sub-roads that include unconventional lanes and sub-roads that do not. The boundaries of the roads differ for different parts. That is, the boundary pair formed by the first boundary and the second boundary can include a first sub-boundary pair and a second sub-boundary pair, where the first sub-boundary pair indicates the boundary of the sub-road that includes unconventional lanes, and the second sub-boundary pair indicates the boundary of the sub-road that does not include unconventional lanes.
[0116] In some implementations, the first road is a one-way road and includes a first lane. When the lane type information indicates that the first lane is an unconventional lane, the area between the first boundary and the second boundary does not include the first lane; or, when the lane type information indicates that the first lane is a regular lane, the area between the first boundary and the second boundary includes the first lane.
[0117] For example, if the first road is the right-hand road shown in Figure 6(a) (i.e., the road between boundary 603 and boundary 601), then the first boundary and the second boundary can be the boundaries corresponding to boundary 602 and boundary 603, respectively, that is, the bicycle lane (i.e., the lane between boundary 601 and boundary 603) is not included between the first boundary and the second boundary.
[0118] In some implementations, the method further includes: inferring the position of at least one lane line in the first road and the boundary of the first intersection based on environmental perception information; inferring the first boundary and the second boundary of the first road, including: inferring the first boundary and the second boundary based on the position of at least one lane line, the boundary of the first intersection, and the type of the first intersection.
[0119] More specifically, when environmental perception information indicates the position of lane lines, point cloud processing or image recognition can be performed on the environmental perception information to determine the position of at least one lane, thereby determining the degree of fit between the lane line and the boundary of the first road. The positions of the inferred first and second boundaries can be corrected based on the degree of fit between the lane line and the boundary of the first road. For example, if the position of the inferred lane line is not between the first and second boundaries, or if the position of the lane line deviates significantly—for example, as shown in Figure 9(a), the positional relationship between the lane line at position 2 and the first and second boundaries—it can be considered that the degree of fit between the lane line and the boundary of the first road is poor. In contrast, as shown in Figure 9(a), the positional relationship between the lane line at position 1 and the first and second boundaries can be considered that the degree of fit between the lane line and the boundary of the first road is good. When the degree of fit between the lane line and the boundary of the first road is poor, and the confidence level of the inferred lane line position is high, the boundary of the first road can be corrected based on the position of the lane line.
[0120] Furthermore, point cloud processing or image recognition can be performed on the environmental perception information to determine the boundary of the first intersection, and then the degree of fit between the boundary lane lines of the first intersection and the boundary of the first road can be determined. The positions of the inferred first and second boundaries can be corrected based on the degree of fit between the boundaries of the first intersection and the first road. For example, when the inferred intersection boundary deviates significantly from the boundary of the first road, such as the positional relationship between the intersection boundary and the first and second boundaries at position 1 as shown in Figure 9(b), the degree of fit between the intersection boundary and the boundary of the first road can be considered poor. In contrast, the positional relationship between the intersection boundary and the first and second boundaries at position 2 as shown in Figure 9(b) can be considered to indicate a better degree of fit between the intersection boundary and the boundary of the first road. When the degree of fit between the determined intersection boundary and the boundary of the first road is poor, and the confidence level of the inferred intersection boundary position is high, the boundary of the first road can be corrected based on the position of the lane lines.
[0121] In practical implementation, the positions of lane lines and / or intersection boundaries can be corrected based on the inferred road boundaries to improve the fit or self-consistency between road boundaries, lane lines, and intersection boundaries. For example, for the inferred road boundary and lane line positions, the confidence level of the inferred lane line positions is higher for positions closer to vehicles. Therefore, for road elements closer to vehicles, the position of the road boundary can be corrected based on the lane line position; conversely, the confidence level of the inferred road boundary is higher for positions farther from vehicles. Therefore, for road elements farther from vehicles, the position of the lane line can be corrected based on the road boundary position, thereby improving the fit between various road elements in the road.
[0122] In some implementations, the first road is connected to the first intersection boundary of the first intersection, and the environmental perception information also indicates at least one boundary including the second road. The second road is connected to the second intersection boundary of the first intersection, and the second road is the road that the vehicle travels to its destination. The method also includes: inferring the boundary of the second road based on the environmental perception information, the type of the first intersection, the first boundary, and the second boundary.
[0123] For example, the intersection shown in Figure 10 can be considered as an example of a first intersection. If road 1 is an example of a first road, then the boundary A of the intersection is an example of the boundary of the first intersection. Further, if the vehicle's direction of travel at the intersection is straight, then road 3 is an example of a second road, and the boundary B of the intersection is an example of the boundary of the second intersection.
[0124] In some implementations, the boundary of the first road obtained through reasoning can be corrected based on the topological relationship between the first road and the second road.
[0125] For example, in practical implementation, the boundaries of the first and second roads obtained through inference are tracked spatially and temporally to ensure that the road boundaries inferred from multi-frame environmental perception information remain stable in terms of geometry and road topology. Thus, if the road boundaries inferred from a single frame of environmental perception information are incomplete or inaccurate due to sensor obstruction, the inference results based on multi-frame environmental perception information can correct and compensate for anomalies caused by single-frame environmental perception information, thereby reducing the probability of anomalies in downstream systems (such as vehicle path planning systems and control systems).
[0126] In some implementations, environmental perception information is input into a first model to obtain a first boundary and a second boundary; wherein, the first model is trained based on multiple sets of road boundary ground values, each set of road boundary ground values indicates the actual location of the road boundary, and each set of road boundary ground values includes the boundaries on both sides of the road.
[0127] For example, the first model can be the neural network model shown in Figure 7 above, or it can be other neural network models, such as other end-to-end neural network models. The road boundary ground truth can be the road boundary ground truth involved in the aforementioned method 300. The method for obtaining the road boundary ground truth and the method for training the neural network model using the road boundary ground truth can be referred to the description in method 300, and will not be repeated here.
[0128] The road element detection method provided in this application embodiment can infer paired boundaries of the first road based on the topological relationship between the first intersection and the first road, as well as the type of the first intersection, when the environmental perception information only indicates one boundary of the first road. This improves the scalability of the vehicle's sensors in terms of perception accuracy and perception range. In other words, it can infer road elements that the sensors have not perceived, thereby improving the completeness and accuracy of road boundary recognition, and thus improving vehicle driving safety and traffic efficiency.
[0129] In the various embodiments of this application, unless otherwise specified or in case of logical conflict, the terminology and / or descriptions between the various embodiments are consistent and can be referenced by each other. Technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationships.
[0130] The methods provided by the embodiments of this application have been described in detail above with reference to Figures 1 to 10. The apparatus provided by the embodiments of this application will now be described in detail with reference to Figures 11 and 12. It should be understood that the descriptions of the apparatus embodiments correspond to the descriptions of the method embodiments; therefore, any content not described in detail can be referred to the method embodiments above, and for the sake of brevity, will not be repeated here.
[0131] Figure 11 shows a schematic block diagram of a road element detection device 2000 provided in an embodiment of this application. The device 2000 may include units for executing the methods described in the foregoing embodiments. Furthermore, each unit in the device 2000 implements a corresponding process of the above method embodiments. The device 2000 includes an acquisition unit 2010, which can be used to implement corresponding data acquisition or transmission / reception functions. The device 2000 also includes a processing unit 2020, which can be used to implement corresponding processing functions.
[0132] Optionally, the device 2000 further includes a storage unit, which can be used to store instructions and / or data. The processing unit 2020 can read the instructions and / or data in the storage unit so that the device can perform the relevant actions in the aforementioned method embodiments.
[0133] It should be understood that the specific process of each unit performing the above-mentioned corresponding steps has been described in detail in the above method embodiments, and will not be repeated here for the sake of brevity.
[0134] It should also be understood that the device 2000 described herein is embodied in the form of a functional unit. The terms “module” or “unit” may refer to application-specific ASICs, electronic circuits, processors (e.g., shared processors, proprietary processors, or group processors) and memory for executing one or more software or firmware programs, integrated logic circuits, and / or other suitable components that support the described functions.
[0135] The apparatus in this embodiment has the function of implementing the corresponding steps in the aforementioned method. The function can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions; for example, the acquisition unit 2010 can be replaced by a transceiver, and other units, such as the processing unit, can be replaced by a processor, used to execute the relevant processing operations in each method embodiment.
[0136] Exemplarily, the acquisition unit 2010 and processing unit 2020 can be disposed in the vehicle 100 shown in FIG. 1, or they can also be disposed in the system shown in FIG. 2. More specifically, the acquisition unit 2010 and processing unit 2020 can be disposed in the element extraction module 223. Exemplarily, the operations performed by the acquisition unit 2010 and processing unit 2020 can be performed by a single processor, or they can be performed by different processors. In specific implementation, the one or more processors can be processors disposed in the vehicle 100 shown in FIG. 1; or, the device 2000 can be a chip disposed in the vehicle 100.
[0137] In the specific implementation process, the units in the above device can be fully or partially integrated together, or they can be implemented independently. In one implementation, these units are integrated together and implemented in the form of a system-on-a-chip (SoC).
[0138] Figure 12 is another schematic block diagram of the road element detection device provided in an embodiment of this application. The device 2100 shown in Figure 12 may include a processor 2110, a transceiver 2120, and a memory 2130. The processor 2110, transceiver 2120, and memory 2130 are connected via internal connection paths. The memory 2130 is used to store instructions, and the processor 2110 is used to execute the instructions stored in the memory 2130 to implement the methods in the above embodiments. Optionally, the memory 2130 may be coupled to the processor 2110 via an interface or integrated with the processor 2110.
[0139] It should be noted that the transceiver 2120 mentioned above may include, but is not limited to, transceiver devices such as input / output interfaces, to realize communication between device 2100 and other devices or communication networks.
[0140] Memory 2130 can be volatile memory and / or non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM). For example, RAM can be used as an external cache. By way of example and not limitation, RAM includes various forms such as: static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).
[0141] Transceiver 2120 uses transceiver devices, such as but not limited to transceivers, to enable communication between device 2100 and other devices or communication networks to receive / send data / information for implementing the methods in the above embodiments.
[0142] This application also provides a vehicle that includes the device 2000 or device 2100 described in the above embodiments.
[0143] This application also provides a computer program product, which includes computer program code. When the computer program code is run on a computer, it causes the computer to implement the methods described in the above embodiments of this application.
[0144] This application also provides a computer-readable storage medium storing computer instructions that, when executed on a computer, cause the computer to implement the methods described in the above embodiments of this application.
[0145] This application also provides a chip, including circuitry, for performing the methods described in the above embodiments of this application.
[0146] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0147] In the description of the embodiments of this application, unless otherwise stated, " / " means "or", for example, A / B can mean A or B; "and / or" in this document describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. In this application, "at least one" means one or more, and "more" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.
[0148] The use of prefixes such as "first" and "second" in this application embodiment is solely for distinguishing different descriptive objects and does not limit the position, order, priority, quantity, or content of the described objects. The use of ordinal numbers and other prefixes to distinguish descriptive objects in this application embodiment does not constitute a limitation on the described objects. The description of the described objects is found in the claims or the context of the embodiments, and the use of such prefixes should not constitute unnecessary restrictions.
[0149] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.
[0150] In the various embodiments of this application, unless otherwise specified or in case of logical conflict, the terminology and / or descriptions between the various embodiments are consistent and can be referenced by each other. Technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationships.
[0151] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0152] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.
[0153] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A method for detecting road elements, characterized in that, include: Acquire environmental perception information, which at least indicates a boundary of the first road where the vehicle is located, and a first intersection connecting to the first road; Based on the environmental perception information and the type of the first intersection, the first boundary and the second boundary of the first road are inferred based on the first model. The area between the first boundary and the second boundary is a one-way road.
2. The method according to claim 1, characterized in that, The reasoning of the first boundary and the second boundary of the first road includes: Based on the boundary of the first road indicated by the environmental perception information, infer the first boundary; Determine the width of the first road based on the type of the first intersection; The second boundary is inferred based on the width of the first boundary and the width of the first road.
3. The method according to claim 1 or 2, characterized in that, The method further includes: Obtain lane type information, which indicates the type of at least one lane in the first road; The reasoning of the first boundary and the second boundary of the first road includes: Based on the environmental perception information, the lane type information, and the type of the first intersection, the first boundary and the second boundary are inferred.
4. The method according to claim 3, characterized in that, The first road is a one-way road and includes a first lane. When the lane type information indicates that the first lane is an unconventional lane, the area between the first boundary and the second boundary does not include the first lane; or, when the lane type information indicates that the first lane is a regular lane, the area between the first boundary and the second boundary includes the first lane.
5. The method according to any one of claims 1 to 4, characterized in that, The method further includes: Based on the environmental perception information, the position of at least one lane line in the first road and the boundary of the first intersection are inferred. The reasoning of the first boundary and the second boundary of the first road includes: Based on the position of the at least one lane line, the boundary of the first intersection, and the type of the first intersection, the first boundary and the second boundary are inferred.
6. The method according to any one of claims 1 to 5, characterized in that, The first road connects to the first intersection boundary of the first intersection, and the environmental perception information further indicates at least one boundary including the second road, the second road connects to the second intersection boundary of the first intersection, and the second road is the road taken by the vehicle to its destination. The method further includes: Based on the environmental perception information, the type of the first intersection, the first boundary, and the second boundary, the boundary of the second road is inferred.
7. The method according to any one of claims 1 to 6, characterized in that, The first model is trained based on multiple sets of road boundary ground values. Each set of road boundary ground values indicates the actual location of the road boundary, and each set of road boundary ground values includes the boundaries on both sides of the road.
8. The method according to claim 7, characterized in that, The true value of the road boundary is obtained by dividing the reference road based on intersections and / or unconventional lanes, and the reference road indicates the actual boundary position of the road.
9. The method according to any one of claims 1 to 8, characterized in that, The first model is an end-to-end neural network model.
10. A road element detection device, characterized in that, include: An acquisition unit is used to acquire environmental perception information, which at least indicates a boundary of the first road where the vehicle is located, and a first intersection connecting to the first road. The processing unit is configured to infer the first boundary and the second boundary of the first road based on the environmental perception information and the type of the first intersection, using a first model. The area between the first boundary and the second boundary is a one-way road.
11. The apparatus according to claim 10, characterized in that, The processing unit is used for: Based on the boundary of the first road indicated by the environmental perception information, infer the first boundary; Determine the width of the first road based on the type of the first intersection; The second boundary is inferred based on the width of the first boundary and the width of the first road.
12. The apparatus according to claim 10 or 11, characterized in that, The acquisition unit is also used for: Obtain lane type information, which indicates the type of at least one lane in the first road; The processing unit is also used for: Based on the environmental perception information, the lane type information, and the type of the first intersection, the first boundary and the second boundary are inferred.
13. The apparatus according to claim 12, characterized in that, The first road is a one-way road and includes a first lane. When the lane type information indicates that the first lane is an unconventional lane, the area between the first boundary and the second boundary does not include the first lane; or, when the lane type information indicates that the first lane is a regular lane, the area between the first boundary and the second boundary includes the first lane.
14. The apparatus according to any one of claims 10 to 13, characterized in that, The processing unit is also used for: Based on the environmental perception information, the position of at least one lane line in the first road and the boundary of the first intersection are inferred. Based on the position of the at least one lane line, the boundary of the first intersection, and the type of the first intersection, the first boundary and the second boundary are inferred.
15. The apparatus according to any one of claims 10 to 14, characterized in that, The first road connects to the first intersection boundary of the first intersection, and the environmental perception information further indicates at least one boundary including the second road, which connects to the second intersection boundary of the first intersection, and the second road is the road the vehicle travels to its destination. The processing unit is further configured to: Based on the environmental perception information, the type of the first intersection, the first boundary, and the second boundary, the boundary of the second road is inferred.
16. The apparatus according to any one of claims 10 to 15, characterized in that, The first model is trained based on multiple sets of road boundary ground values. Each set of road boundary ground values indicates the actual location of the road boundary, and each set of road boundary ground values includes the boundaries on both sides of the road.
17. The apparatus according to claim 16, characterized in that, The true value of the road boundary is obtained by dividing the reference road based on intersections and / or unconventional lanes, and the reference road indicates the actual boundary position of the road.
18. The method according to any one of claims 1 to 8, characterized in that, The first model is an end-to-end neural network model.
19. A road element detection device, characterized in that, include: A processor for executing a computer program stored in memory to cause the apparatus to perform the method as described in any one of claims 1 to 9.
20. A computer-readable storage medium, characterized in that, It stores instructions that, when executed by a processor, implement the method as described in any one of claims 1 to 9.
21. A chip, characterized in that, The chip includes circuitry for performing the method as described in any one of claims 1 to 9.
22. A computer program product, characterized in that, The computer program product includes: computer program code, which, when executed by a processor, implements the method as described in any one of claims 1 to 9.
23. A vehicle, characterized in that, Includes the apparatus as described in any one of claims 10 to 19, or the computer-readable storage medium as described in claim 20, or the chip as described in claim 21, or the vehicle is equipped with the computer program product as described in claim 22.