Method, cloud server and vehicle for generating high-definition map

Abstract

The present disclosure relates to a method, a cloud server and a vehicle for generating a high definition map. A computer implemented method of generating a map is provided, comprising the steps of: obtaining sensor based data of a sensor unit of at least one vehicle and positioning data related to a positioning unit of the at least one vehicle; determining a first trajectory of the at least one vehicle using the positioning data and determining a plurality of second trajectories of other vehicles using the sensor based data; and aggregating the first trajectory and / or the plurality of second trajectories to generate an aggregated trajectory in a grid map representation of the map.

Description

Technical Field

[0001] This disclosure relates to a computer implementation method for generating maps, and a cloud server and vehicle for making improved ADAS decisions based on maps. Background Technology

[0002] High-definition (HD) maps are essential in Advanced Driver Assistance Systems (ADAS) or Autonomous Driving (AD). HD maps typically refer to maps accurate to the centimeter level, containing far more detail (e.g., road signs, traffic signs, road obstacles) than regular automotive maps (e.g., those used for conventional car navigation), and are therefore suitable for ADAS and / or AD. HD maps can be generated based on data from one or more sensors (e.g., radar, digital camera, lidar data) and GPS data, but can also be generated based on aerial imaging, such as that produced by unmanned or low-Earth orbit satellites.

[0003] While a vehicle's sensor system (e.g., consisting of cameras and radar) is responsible for real-time perception of the environment, HD maps provide information about the road system, the geometry ahead, and the functional significance of driving lanes; for example, they enable the understanding that a particular lane is a dedicated bus lane. Therefore, they define where vehicles are permitted to travel, the location of traffic signs, and, for example, speed limits.

[0004] These HD maps can be set up as a framework in which vehicles (ADAS and / or AD) can move. Furthermore, HD maps can be considered a backup solution when, for example, a real-time system malfunctions or is obstructed.

[0005] Creating HD maps is a challenging task. Typically, map providers utilize multiple surveillance vehicles equipped with high-cost sensors such as high-resolution cameras and LiDAR. These vehicles then travel across all roads in the relevant area to map and aggregate the data offline. Sometimes, even additional information is added manually. The problem is exacerbated by the relatively rapid mapping of HD output data as the environment and road geometry change over time. Therefore, creating and preserving HD maps is a challenging task.

[0006] HD maps can be created or generated in several ways. They can be manually annotated, which is often very expensive due to the human annotation effort. Therefore, this manual method is not scalable, as HD maps are needed almost everywhere and they must be kept up-to-date.

[0007] Some traditional methods also use GPS signals from multiple devices as crowdsourced data ([1]: Cao et al., “From gps traces to a routable road map.” 17th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, 2009, pp. 3-12; [2]: Tang et al., “Lane-level road information mining from vehicle gps trajectories based on Bayesian classification, ISPRS International Journal of Geographic Information, 2015, 4(4):2660–2680), but these traditional methods also have drawbacks, as they provide only a limited amount of information or focus on road maps that are less accurate than HD maps for planning routes. In addition, the methods in [1] and [2] are not scalable because they only consider GPS signals.

[0008] Other conventional methods train deep neural networks (DNNs) based on various sensor data to predict road geometry ([3]: “Learning to map by discovering lane topology” by Homayounfar et al., Proceedings of the IEEE / CVF International Conference on Computer Vision, Vol. 2911-2920, 2019; [4]: ​​“Lane graph estimation for scene understanding in urban driving” by Zürn et al., arXiv preprint arXiv:2105.00195, 2021). In [4], DNNs are trained based on sensor input data with various partial processing to infer road geometry in a graph representation. However, this method requires annotated HD map data. The method in [3] follows a similar approach to that in [4], but primarily uses lidar sensor data. This work focuses on the prediction of lane boundaries and bifurcation and merging on highways, but requires training of the DNN, ground reality, and therefore early HD map data.

[0009] The methods presented in [5]: “Anytime lane-level intersection estimation based on trajectories of other traffic participants” by Meyer et al., IEEE Intelligent Transportation Systems Conference (ITSC) 2019, pp. 3122-3129, IEEE, 2019, and [6]: “Fast lane-level intersection estimation using markov chain Monte Carlo sampling and b-spline refinement” by Meyer et al. (IEEE Intelligent Vehicles Workshop (IV) 2020, pp. 71-76, IEEE, 2020) use the trajectories of other road users detected and tracked for lane estimation. Both works focus on intersection scenarios and represent lanes as lane boundaries. Trajectories can be based on cameras, radar, or lidar. Instead of using DNNs, the authors use probabilistic generative models, specifically the two-stage Markov Chain Monte Carlo sampling approach. In addition to [5], in [6], the authors introduce a nonlinear least squares formula to improve the results. The main task is to evaluate the simulation data, especially to simulate vehicle trajectories.

[0010] As already mentioned, it is important not only to create HD maps but also to keep them up to date. Solutions to this are given, for example, in [7]: Pannen et al., “How to keep hd maps for automated driving up to date”, IEEE International Conference on Robotics and Automation (ICRA) 2020, pp. 2288-2294, IEEE, 2020; [8]: Kim et al., “Hd map update for autonomous driving with crowdsourced data”, IEEE Communications on Robotics and Automation, 6(2): 1895-1901, 2021. Traditional systems are usually only applicable to specific environments; they usually do not take into account complex situations such as detours, intersections, etc. Therefore, there is a need for a robust and simpler way to generate HD lane maps that can be used in any traffic environment or street scenario and can improve ADAS and / or AD. Summary of the Invention

[0011] According to a first aspect of this disclosure, a computer implementation method for generating a map includes the following steps: acquiring sensor-based data of a sensor unit of at least one vehicle and positioning data associated with a positioning unit of the at least one vehicle; using the positioning data to determine a first trajectory of the at least one vehicle, and using the sensor-based data to determine a plurality of second trajectories of other vehicles; and aggregating the first trajectory and / or the plurality of second trajectories to generate aggregated trajectories in a grid map representation of the map.

[0012] According to a second aspect of this disclosure, the method further includes: extracting the trajectory of the lane center of the map from the aggregated trajectory.

[0013] According to a third aspect of this disclosure, the method further includes: classifying the first trajectory and the plurality of second trajectories according to a predetermined set of different driving directions, and generating a plurality of driving direction-specific clustered trajectories.

[0014] According to a fourth aspect of this disclosure, the aggregation trajectory is associated with a density map (D), and the direction-specific aggregation trajectory is associated with the direction-specific density map (D). d The density map and the driving direction density map are associated with each other and are generated using data collected from the first trajectory and / or the plurality of second trajectories.

[0015] According to the fifth aspect of this disclosure, the extraction of the trajectory of the lane center is performed by selecting an aggregated trajectory value that is greater than or equal to a predetermined threshold.

[0016] According to the sixth aspect of this disclosure, based on the processed density map (s(D)) and / or the processed driving direction density map (s(D)) d The process is used to extract the trajectory of the lane center, and the processed density map (s(D)) and the processed driving direction density map (s(D)) are generated. d This is achieved by using density maps (D) and driving direction density maps (D). d The function is used to determine the corresponding local density maximum in the direction perpendicular to the specific aggregation trajectory and the direction of travel.

[0017] According to the seventh aspect of this disclosure, the trajectory of the lane center is extracted by tracking an ordered path with local density maximum.

[0018] According to the eighth aspect of this disclosure, the trajectory of the lane center is post-processed to connect the ends of the extracted trajectory of the lane center.

[0019] According to a ninth aspect of this disclosure, the method further includes: representing a clustering trajectory in a directed graph representation of an HD map including the direction of travel.

[0020] According to a tenth aspect of this disclosure, the directed graph representation further includes one or more representations of vehicle density, lane bifurcation, lane merging, and lane boundaries.

[0021] According to the eleventh aspect of this disclosure, the method further includes: using the sensor-based data to determine a plurality of third trajectories of pedestrians; and aggregating the plurality of third trajectories to generate aggregated pedestrian trajectories in the grid map representation of the map.

[0022] According to a twelfth aspect of this disclosure, the plurality of second trajectories of the other vehicles are determined by distinguishing other vehicles according to vehicle categories and determining a set of vehicle category-specific second trajectories; and the method further includes: aggregating the plurality of vehicle category-specific second trajectories to generate vehicle category-specific aggregated trajectories in the grid map representation of the HD map.

[0023] According to a thirteenth aspect of this disclosure, the method further includes storing the determined second trajectory together with metadata associated with the second trajectory and / or the clustered trajectory.

[0024] According to the fourteenth aspect of this disclosure, the method further includes storing the determined second trajectory together with metadata related to the corresponding detection of the category.

[0025] According to the fifteenth aspect of this disclosure, the grid map of the HD map represents the average velocity information of each grid cell of the HD map.

[0026] According to the sixteenth aspect of this disclosure, the sensor-based data is based on radar sensor data and / or lidar sensor data.

[0027] According to the seventeenth aspect of this disclosure, the positioning data is based on GPG data.

[0028] According to the eighteenth aspect of this disclosure, a cloud server is adapted to perform the methods of any one of the first to seventeenth aspects.

[0029] According to the nineteenth aspect of this disclosure, a vehicle includes: a communication interface configured to receive a map from a cloud server according to the fifteenth aspect of this disclosure; and a control unit configured to make advanced driving and safety decisions based on the received map.

[0030] According to the twentieth aspect of this disclosure, the control unit is configured to use only aggregated trajectories in the map that have been aggregated by using a minimum number of individual trajectories and / or that have been aggregated by using a minimum number of trajectories determined over a specific amount of time in the past.

[0031] According to the twenty-first aspect of this disclosure, the control unit is configured to output control commands to the vehicle to follow one of the clustered trajectories from a received map.

[0032] According to the twenty-second aspect of this disclosure, the control unit is further configured to output control commands to the vehicle to follow one of the convergence trajectories, provided that the convergence trajectory is accurate.

[0033] According to the twentieth aspect of this disclosure, the control unit is further configured to determine a cornering speed recommendation based on the received map.

[0034] According to the twenty-fourth aspect of this disclosure, a system includes a cloud server according to the eighteenth aspect and a plurality of vehicles according to any one of the nineteenth to twenty-third aspects. Attached Figure Description

[0035] Figure 1 The vehicle system and cloud server according to the implementation method are shown.

[0036] Figure 2 An apparatus for acquiring radar-based sensor data according to an embodiment is shown.

[0037] Figure 3 The vehicle and detection area according to the implementation method are shown.

[0038] Figure 4 A programmable signal processing apparatus according to an embodiment is shown.

[0039] Figure 5 A flowchart of a method for generating maps based on clustered trajectories according to an embodiment is shown.

[0040] Figure 6 The bounding box detection and tracking, as well as trajectory determination (represented in a map representation), are shown according to an implementation method.

[0041] Figure 7 The results of the combined aggregation and extraction process are shown.

[0042] Figure 8 The directional density map D derived from the global density map D is shown. d Examples.

[0043] Figure 9 A method for extracting the trajectory of the lane center according to an embodiment is shown.

[0044] Figure 10 The iteration of pre-extraction according to the implementation method is shown.

[0045] Figure 11An example of a cutting connection process according to an embodiment is shown.

[0046] Figure 12 An example of connecting the extracted trajectory of the lane center according to an implementation method is shown.

[0047] Figure 13 A flowchart illustrating a method for determining a vehicle category-specific aggregation trajectory according to an implementation method is shown.

[0048] Figure 14 A flowchart of a method implemented by a vehicle according to an embodiment is shown. Detailed Implementation

[0049] Embodiments of the present disclosure will now be described with reference to the accompanying drawings. Numerous specific details are set forth in the following detailed description. These specific details are intended only to provide a thorough understanding of the various described embodiments. Furthermore, although the terms first, second, etc., may be used to describe various elements, these elements are not limited by these terms. These terms are used only to distinguish one element from another.

[0050] This disclosure teaches how to generate a map (HD map) based on the aggregation of individual trajectories. Here, since ADAS vehicles perceive other road users around them (e.g., cars, trucks, buses, bicycles, motorcycles, pedestrians), and each road user is associated with an individual trajectory (since each road user leaves a trail behind, i.e., the path they travel), the trajectory is a compressed fragment of information about the perceived road geometry, and the collection or aggregation of individual trajectories of multiple road users provides an accurate source of information.

[0051] Figure 1 A system of vehicle 10 and cloud server 20 according to an embodiment is shown. Vehicle 10 (e.g., an ADAS fleet) is equipped with a communication interface to establish a communication connection with cloud server 20. This communication connection can be implemented in different forms; such as... Figure 1 As shown, vehicle 10 may have, for example, an over-the-air (OTA) interface with base station 30 (e.g., a radio interface according to wireless communication standards such as LTE, 4G, 5G, 6G). Base station 30 provides a communication connection to cloud server 20 (e.g., via a core network, the Internet, etc.). Based on this communication connection, data related to various trajectories of vehicle 20 can be uploaded to cloud server 20, and map information or other ADAS-related information can be downloaded from cloud server 20 to vehicle 10, for example, in the form of Software as a Service (SaaS). Here, the underlying map is continuously updated by aggregating various trajectories and deriving additional high-level information from the aggregated trajectories. The resulting up-to-date and accurate map can then be downloaded from cloud server 20 to vehicle 10 to, for example, improve existing ADAS features.

[0052] Vehicle 10 (especially ADAS vehicles) is equipped with a wide range of sensor units (e.g., cameras and radar and / or lidar) for environmental perception. These sensors allow vehicle 10 to perceive its environment and, for example, detect lane markings, traffic signs, and other road users, including its dynamic characteristics (position and speed relative to the ADAS vehicle). This perception software (SW) forms the basis of modern ADAS systems and will exist in every ADAS vehicle (with varying degrees of maturity and quality). Based on this, ADAS vehicles are typically equipped with L2+ capabilities, i.e., they can perform lane keeping and distance control.

[0053] Figure 2 An apparatus 100 for generating maps according to an exemplary embodiment of the present disclosure is shown. The apparatus 100 may be disposed in a vehicle 20, and preferably mounted on the vehicle 20 facing the direction of travel D of the vehicle 20. Those skilled in the art will understand that the apparatus 100 is not necessarily facing the direction of travel; the apparatus 100 may also face laterally or rearward. The apparatus 100 may be part of a radar sensor, radar module, or radar system. The apparatus 100 may also be part of a lidar sensor, lidar module, or lidar system. The apparatus 100 may also be a device for acquiring and processing data from multiple different sensors (such as radar data and lidar data). The vehicle 10 may also have more than one such apparatus 100.

[0054] The device 100 can be provided as part of the driver assistance system 200 of the vehicle 10, or as... Figure 2 The system interacts with the driver assistance system 200 of vehicle 10. The driver assistance system 200 may include advanced driver assistance system (ADAS) functions or autonomous driving functions.

[0055] Vehicle 10 can be any land vehicle that moves by mechanical power. The accompanying drawings illustrate vehicle 10 as a car equipped with device 100. However, this disclosure is not limited thereto. Therefore, device 100 can also be installed on, for example, vans, trucks, motorcycles, buses, robots, etc.

[0056] Device 100 may have multiple detection areas, for example, oriented to have such... Figure 3 The forward detection area 111F, left detection area 111L, and / or right detection area 111R are shown in the plan view. As a non-limiting example, an object can be considered to be around the vehicle 10 at least when it is located within one or more of the plurality of detection areas of the device 100.

[0057] like Figure 2As shown, device 100 includes an acquisition unit 120 and a processing unit 130. Additionally, device 100 may optionally include one or more sensors or sensor units (e.g., radar and / or lidar sensors) 110, but the one or more sensor units 110 may also be disposed separately from device 100.

[0058] The acquisition unit 120 can be configured (e.g., programmed) to acquire sensor-based data from at least one sensor unit 110 of the vehicle 10, as well as positioning data associated with the vehicle 10's positioning units (e.g., GPS unit, DPGS unit, not shown). The sensor-based data may be radar data (from a radar sensor) and / or lidar data (from a lidar sensor). Note that the vehicle's sensor-based data and positioning data are real-world data, not simulated data. Acquiring sensor-based data from the vehicle's sensor unit 110 can be performed via an in-vehicle data communication interface, such as CAN bus communication or Ethernet to the vehicle 10's area controller or domain controller.

[0059] Processing unit 130 can be configured (e.g., programmed) to determine a first (individual) trajectory of vehicle 10 itself using positioning data (e.g., using GPS data), and to determine multiple second (individual) trajectories of other vehicles or other objects around vehicle 10 (within the detection area of ​​the sensor unit) using acquired sensor-based data. A trajectory can generally be considered as the path or trajectory (a set of coordinates in space) of a vehicle (or any other object) over time in a global coordinate system (GCS). Trajectory information can be stored in the memory of vehicle 10, for example, along with additional metadata such as recording time, the speed of the tracked object, etc. Here, the first path is associated with the individual path of vehicle 10 itself (hereinafter also referred to as the ego vehicle). In other words, the ego vehicle's trajectory or path t ego This can be provided by the vehicle's location (using GPS data stream) and (known) dimensions.

[0060] Those skilled in the art will understand that the first trajectory may be a set of GPS or DGPS positioning data for vehicle 10. The second trajectory is associated with individual trajectories or paths of other observed road users detected in the sensor detection area of ​​the self-vehicle 10. Here, since sensor-based data (due to radar and / or lidar detection) is typically associated with relative position data about the self-vehicle, the positioning data (first trajectory) can be advantageously used to determine multiple second trajectories in the GCS.

[0061] Processing unit 130 can also be configured (e.g., programmed) to aggregate a first trajectory (of the ego vehicle) with multiple second trajectories of other vehicles (other road users), or to aggregate multiple second trajectories of other vehicles (which may also include other road users such as motorcycles, bicycles, etc.) to generate multiple aggregated trajectories. Generating aggregated trajectories can generally be viewed as collecting individual trajectories together, in other words, as a sum of individual trajectories. Those skilled in the art will understand that aggregating individual trajectories together allows for the determination (statistical averaging) of the trajectories already acquired by the majority of road users, and where individual deviations are averaged. This further improves the accuracy of aggregated trajectories because the ego vehicle's first trajectory typically has better detection accuracy compared to sensor-based determination of second trajectories of other vehicles (e.g., because GPS positioning can be performed with high precision).

[0062] Processing unit 130 can perform aggregation by associating the trajectory of its own vehicle with a higher statistical weight than that of other detected vehicles. Furthermore, when performing aggregation, detected vehicles of different sizes can also be associated with different statistical weights, because a large truck is more likely to accurately follow a lane (e.g., lane center) compared to a smaller vehicle or motorcycle that has some flexibility in driving more towards the center or side of the road or lane.

[0063] Here, the generated aggregation trajectories can be included in a grid map representation of a map (HD map). That is, a map with a predetermined spatial resolution (e.g., in a 2D Cartesian coordinate system) can be subdivided into multiple individual grid cells of a predetermined size, such as each grid cell covering a size of 10cm × 10cm, and the aggregation trajectories are included in that map. Thus, data can be provided by a single ego vehicle, and detailed information about multiple lanes can be aggregated from that data to create an HD map.

[0064] Those skilled in the art will understand that aggregated trajectories can be generated in any traffic environment and in any geographical area, and the corresponding maps are low-cost and reliable maps.

[0065] Figure 4 This is a schematic diagram of a programmable signal processing device 300, which can be configured to implement... Figure 2 The device has 100 functions.

[0066] The signal processing device 300 has an interface module 310 that provides means for transmitting and receiving information, such as one or more antennas or wired connections, for example, for providing a communication connection to the cloud server 20. The signal processing device 300 also includes: a processor 320, such as a CPU, for controlling the programmable signal processing device 300 to perform the functions of the device 100; a working memory 330, such as a random access memory; and an instruction storage unit 340 for storing a computer program 345 having computer-readable instructions that, when executed by the processor 320, cause the processor 320 to perform the functions of the device 100.

[0067] The instruction storage unit 340 may include a ROM, such as an electrically erasable programmable read-only memory (EEPROM) or flash memory, which is preloaded with computer-readable instructions. Alternatively, the instruction storage unit 340 may include RAM or a similar type of memory, and the computer-readable instructions may be input thereto from a computer program product (e.g., a computer-readable storage medium 350, such as a CD-ROM) or a computer-readable signal 360 carrying the computer-readable instructions.

[0068] Alternatively, device 100 may be implemented in non-programmable hardware such as application-specific integrated circuits (ASICs), or in any other suitable manner using any appropriate combination of hardware and software components.

[0069] As described above, this disclosure provides various techniques for generating maps in a more accurate and reliable manner.

[0070] The processing unit 300 can also be implemented in the cloud server 20 to achieve... Figure 2 The functions of device 100. In other words, the functions of acquisition unit 120 and processing unit 130 can also be implemented in a cloud server. Figure 5 A flowchart is shown of a method for generating a map based on a clustering trajectory, according to another embodiment implemented in base station 20.

[0071] According to step S110, cloud server 20 acquires sensor-based data and positioning data from multiple vehicles 10 as described above. This acquisition can be performed via over-the-air (OTA) communication between the vehicles 10 and cloud server 20, for example via I / F 310 to one or more base stations 30, as described above.

[0072] According to step S130, the cloud server 20 (through the processor 320 that implements the functions of the processing unit 130) can then use the positioning data to determine the first trajectory of multiple self-vehicles, and for each of the multiple self-vehicles, determine multiple second trajectories of other detected vehicles or objects in the detection area of ​​the self-vehicle.

[0073] A preferred implementation for determining the second trajectory follows detection and tracking using LiDAR-based sensor data. That is, the autonomous vehicle is equipped with a LiDAR-based sensor system that allows for 3D object detection and tracking, and also with a GPS unit to track the vehicle's position. However, this is not limiting, and vehicle detection and tracking can also be performed using radar-based data (Manjunath et al., "Radar Based detection and Tracking for Autonomous Driving", 2018 IEEE MTT-S International Conference on Intelligent Mobile Microwave, DOI:10.1109 / ICMIM.2018.8443497).

[0074] Here, a 3D bounding box detector and tracker (due to LiDAR-based sensor data and / or radar-based sensor data) can be used to identify road users across multiple detection frames. That is, 3D object detection and tracking is applied to sensor-based data. The detection of another vehicle (e.g., a car, truck, bus, etc.) can be represented by a 3D bounding box (e.g., a cuboid) indicating the vehicle's dimensions (e.g., 2m (width) × 4m (length) × 2m (height)), and the corresponding position of the bounding box can be transformed to a global coordinate system (GCS) over multiple times or detection frames. The orientation of the bounding box (cube) can also be determined, thus indicating which side of the bounding box belongs to the front of the corresponding vehicle, thereby indicating the direction of travel. The detection result (cube) can then be recorded along with the direction of travel of the other (detected) vehicle. If the tracked vehicle is stationary or moving, tracker confidence and flags can also be provided.

[0075] The bounding box (cube) can then be processed into a grid map representation indicating the track or trajectory of another vehicle.

[0076] Figure 6 Examples of bounding box detection and tracking, as well as trajectory determination (represented as a trajectory in a map representation) are shown. Figure 6 The left figure schematically illustrates the detection and tracking cube across three frames. Specifically, once other road users (other vehicles) are detected, position, orientation, and direction of travel can be derived from the vehicle's GPS location and sensor system calibration. In the case of a lidar sensor providing dense spatial data with depth information, the cube from the local coordinate system can be transformed to the global coordinate system.

[0077] The track (path) or trajectory t of the vehicle being inspected D It can be defined as the detected cube c in the corresponding detection frame j = 1, ..., p j The trajectory, i.e., t={c1,…,cp}, where p is the last detected frame.

[0078] Each trajectory t D Meta-information can be used to enrich the data, such as the trajectory confidence value t. conf ∈[0,1], the range of distances the cube should be relative to its own vehicle. (set of real numbers), and / or trajectory The Euclidean distance between the first and last cubes. Typically, each trajectory can be associated (e.g., stored) with meta-information as described herein or further below.

[0079] Figure 6 The middle section illustrates the process of transforming the detected and tracked cube into a single trajectory. The entire geographic area covered by the cuboid is treated as a single trajectory; that is, as shown in the diagram... Figure 6 As shown in the right figure, for each trajectory, the overlapping area of ​​the two subsequent cubes is preferably counted only once. Here, Figure 6 The grid in the background represents a map (which can be thought of as a matrix), and the black cells represent the value 1 of the trajectory (right figure).

[0080] By recording locations multiple times and detecting multiple vehicles covering the same location, multiple detections can be aggregated for lane estimation. Therefore, local grid cell regions of the map can have aggregated values, initially set to zero and incremented by one for each trajectory passing through the local grid cell region.

[0081] Furthermore, if the individual trajectory of the ego vehicle (the first trajectory) is also used to generate the aggregated trajectory, then the ego vehicle's location data is used in conjunction with the ego vehicle's known 3D dimensions to generate, for example... Figure 6 The corresponding representation is shown below.

[0082] In addition, according to Figure 5 In step S150, the cloud server 20 (through the processor 320 that implements the functions of the processing unit 130) can then aggregate the first trajectory (of its own vehicle) and multiple second trajectories of other vehicles (other road users), or aggregate multiple second trajectories of other vehicles (other road users) to generate multiple aggregated trajectories.

[0083] For clusters, a given geographic location (local area) can be represented as a grid map, such as... Figure 6 As shown, each grid cell covers a predetermined area, such as a 10cm × 10cm area in a GCS. Density map. It can be defined by clustering over a given spatial location. Density map D can represent the density of moving vehicles.

[0084] Initially, all values ​​of the grid map D are set to 0. Then, for all trajectories (t) covering a given location... D and / or t ego ) Perform iterations. For the movement trajectory t D Each cube c j It can map the detected cubes onto a density map, and when a location (grid) overlaps with a cube, it increases the values ​​of the density map by a predetermined value (e.g., by 1).

[0085] Each value in D can only be increased once by the cube of the current trajectory. That is, when considering, for example... Figure 6 When three cuboids are detected (in the left figure), the value in D increases only once, even in the overlapping area of ​​two subsequently detected cuboids. This prevents the value in D from accumulating incorrectly, such as for a stopped moving vehicle, for example, when it reaches a traffic light.

[0086] After processing the cube of the first trajectory, the value of D can be 0 or 1, see... Figure 6 (Right image). Then, after processing the cube of the second trajectory, the value in D can be 0, 1, 2, etc.

[0087] Since cloud server 20 has acquired the corresponding data from multiple vehicles, it can generate aggregated trajectories from a larger number of individual (first and second) trajectories, and therefore the aggregated trajectories and the resulting maps are more accurate.

[0088] Those skilled in the art should understand that it is not Figure 5 All steps must be performed by the cloud server 20. For example, steps S110 and S130 can be performed by individual autonomous vehicles 10, for instance, in the form of preprocessing, and only information associated with individual tracks is sent to the cloud server to aggregate individual tracks. This reduces the amount of data that must be communicated between the fleet of ADAS vehicles and the cloud server, and thus results in faster updates of the HD map.

[0089] Therefore, the above concept for creating HD maps is based on estimated individual trajectories (second trajectories) of observed road users, and may also include the driving paths of the vehicle itself (first trajectories). Data from multiple sensor records (based on real-world data) can be used, meaning geographic locations can be recorded more than once. However, a single record can also provide sufficient information to estimate individual trajectories. As explained, the trajectories (second trajectories) are associated with vehicles detected and tracked from LiDAR point cloud data and / or radar data. This leads to the aggregation of trajectories and the thereby identified drivable lanes, which vehicles can use to improve ADAS functionality, as will be described in further detail below.

[0090] According to a preferred embodiment, the above-described aggregation process can be followed by an extraction process, in which the trajectory of the lane center is extracted from the aggregation trajectory. The lane center can be considered as the midpoint of the lane that is equal to the left and right space of the driver; therefore, the trajectory of the lane center can follow the midpoint of the lane and can be considered as the optimal path for autonomous driving purposes. The extraction of the lane center can be performed in different ways, such as using the watershed transformation (see, for example, Beucher's "The Watershed Transformation Applied to Image Segmentation", Scanning Microscopy: Vol. 1992: Section 6, Paragraph 28) to cluster and segment the aggregation path.

[0091] exist Figure 7 An example of the results of the combined aggregation and extraction process is shown. Specifically, Figure 7 A portion of the map generated in a complex traffic situation is shown, specifically illustrating the convergence trajectories (generated as described above) and their corresponding directions of travel, extracted lane centers, lane boundaries, and bifurcations and mergings. The convergence trajectories may be displayed alongside indications of traffic density (low to high), for example, derived from the density map described above. Those skilled in the art will understand that high density refers to the increased traffic that can be expected in geographical areas where lanes bifurcate into two or more lanes or merge into a single lane.

[0092] In a preferred embodiment, the first individual trajectory of the ego vehicle and multiple second trajectories of other detected vehicles can be classified according to a predetermined set of different driving directions, and multiple driving direction-specific clustered trajectories can be generated.

[0093] Here, the driving direction can be derived from the cube's orientation (from the 3D object detection and tracking described above). The orientation in the GCS can be determined by the yaw rate. Given, orientation can be divided into four categories (a predetermined set of different driving directions), and density maps related to the four directions can be defined. For simplicity, the direction density map is... d = 1,…,4. Then, as described above, iteration can be performed on all trajectories and cubes, but depending on the cube's orientation, the cube can be projected onto the orientation density map such that D d Includes having the following Cube:

[0094] ·D1:

[0095] ·D2:

[0096] ·D3:

[0097] ·D4:

[0098] This mapping can also depend on metadata, which typically defines the aggregation strategy. For example, if only aggregation occurs near the self-vehicle (i.e., within a specific region around the self-vehicle) (i.e., t... r (low selection) and has high tracker confidence t conf If the trajectory (second trajectory) is the same, then this is a more conservative strategy.

[0099] After mapping the trajectories of road users (second trajectories), the cubes of ego vehicles can also be mapped and aggregated into an orientation density map D. d d = 1, ..., 4. This can be done with or without any metadata about the ego vehicle itself. The weight of the ego vehicle can also be increased, for example, by 2. Since the ego vehicle trajectory can be based on precise positioning, such as dGPS, it can have less positioning uncertainty than trajectories from detected and tracked vehicles (other road users).

[0100] Therefore, as mentioned above, the aggregation trajectory can be associated with the density map D, and the driving direction-specific aggregation trajectory can be associated with the driving direction density map D. d This is correlated, thus using data collected from the first trajectory and multiple second trajectories to generate density maps and driving direction density maps. That is, these maps can be generated by iterating over all the individual trajectories, thereby allowing for the generation of density maps with respect to a specific distance t from the vehicle. r And / or with a specific tracker confidence level t conf Consider a second trajectory (related to detected vehicles or objects in the ego vehicle environment). Additionally, the number of cuboids in each trajectory, i.e., the distance between the first and last cuboids of a single track, can be considered. This allows for further improvement and control over the accuracy and reliability of the generated map.

[0101] Figure 8 The directional density map D, derived from the overall density map D (aggregation trajectory) in the left figure, is shown. d Example of a clustering trajectory (specific to the direction of travel). Those skilled in the art will understand that the overall density map D (clustering trajectory) is composed of four directional density maps D. d The sum of (specific convergence trajectories in the direction of travel).

[0102] The direction-dependent density map described above can be advantageously used in another implementation for extracting lane centers. This is due to observation that a direction-independent density map D may lead to the extraction and connection of opposite lanes, resulting in incorrect mapping. This extraction can be applied to aggregated data in a map representation to produce a structured dataset that represents lanes as lane centers with respect to the direction of travel.

[0103] Figure 9 A method for extracting the trajectory of the lane center according to a preferred embodiment is shown.

[0104] according to Figure 9 Step S210 involves performing preprocessing of the aggregation trajectories. In this preprocessing, only D and D' values ​​greater than or equal to a predetermined threshold are selected. d The clustered trajectory values ​​are used to extract the lane center. This advantageously avoids false positive lanes and / or unwanted detections. Note that the predetermined thresholds for D and D0 are... d They can be different.

[0105] That is, as a preprocessing step for extraction, D can be... d The values ​​in d = 1, ..., 4 are set to 0, where these values ​​are not greater than or equal to the given minimum number of trajectories num. t This parameter can be used to avoid false positives or unwanted detections. For example, if a specific location is recorded 10 times with moderate flow, and only one detection occurs out of those 10 records in a particular area, this could be a false positive. Therefore, the minimum number of tracks, num, is used. t This ensures a certain number of tests related to the records.

[0106] As a further preprocessing step, the processed density map S(D) and / or the processed driving direction density map S(D) can be used as a basis. d The trajectory of the lane center is extracted using the processed density map S(D) and the processed driving direction density map S(D). d ) is obtained by analyzing density map D and driving direction density map D. d The function (smoothing function or filtering function) is used to determine the corresponding local density maximum value in the direction perpendicular to the specific clustering trajectory and the direction of travel.

[0107] This function can be, for example, a convolutional filter that can be applied. For example, density map D and orientation density map D d This can be addressed by applying convolution with stride S=1, kernel size k=20, zero padding, and a fixed kernel weight of 1. Alternatively, zero padding can be used.

[0108] Processed density maps S(D), S(D) d ) can be compared with density maps D, Dd This is more suitable for subsequent extraction of lane centers. For example, if we only have the trajectory of one vehicle, the corresponding value in density map D is 1. In density map S(D), these values ​​will be different: the value of the corresponding center of the trajectory is higher than the value of the station of the trajectory. Therefore, the center of the trajectory can be interpreted as the set of local maxima in an affine subspace orthogonal to the trajectory.

[0109] Since only processed density maps will be used from now on, the term "processed" will be omitted.

[0110] according to Figure 9 In another step S230, pre-extraction or initial extraction of the lane center can be performed. Here, the extraction of the lane center trajectory can be performed by tracking (or following) an ordered path with local density maximum, preferably in the forward and backward directions of the aggregated trajectory.

[0111] For example, the initial extraction (pre-extraction) step S230 can be performed as an iterative process. Specifically, for each density map s(D d For d = 1, ..., 4, the following processing can be performed:

[0112] First, we can select s(D) d The highest value (maximum local maximum) in s(D) is taken as the starting point. d The position of the highest value in the matrix is ​​used to detect the next highest value or the same highest value in the current direction (ordered path). For example, if considering a density map S(D3) representing a direction from south to west, the next position with the highest left, lower left, or lower value or the same value is determined. Pre-extraction with respect to the discrete orientation or direction d is called forward extraction. In backward extraction, processing is performed from the same starting point as in forward processing, but in the opposite direction to d. In this example, values ​​from the current position to the top, top right, and right are evaluated to detect the next highest value.

[0113] Forward and backward extraction stops if at least one of the following conditions is met: 1) the end of the map is reached; 2) the next value will be 0, meaning there is no trajectory information; or 3) the next value is a predetermined amount (e.g., 50%) smaller than the average of the already extracted values. The idea behind the final stopping condition 3) is to avoid lane merging in the case of parallel lanes traveling in the same direction.

[0114] After pre-extraction of the center portion of the first lane, the values ​​in the region surrounding the extracted values ​​are set to 0, and the updated density map s(D) is selected. d The highest value begins on s(D). d The iteration process ends when all values ​​in the expression are 0.

[0115] Figure 10 The first iteration of pre-extraction is shown. Here, the left figure shows S(D3) and the insertion box region associated with the highest density value. Within this box region, the local density maximum follows along the forward (1) and backward (2) directions (second figure). Then, after extracting the first trajectory around the lane center (highlighted as a dashed line in the third figure), the value of the first trajectory around the extracted lane center is set to 0, and the iteration continues until... Figure 10 The center area of ​​the next lane is marked by the black box in the fourth picture.

[0116] according to Figure 9 In step S250, post-processing is performed to cut and connect the extracted lane center trajectory. Here, since the area around the extracted lane center is set to 0 in step S230 during pre-extraction, the end of the extracted lane center may not be connected to the end of the adjacent end of the lane center, see [link to previous steps]. Figure 11 The left image shows the post-processing of cutting and connecting the extracted lane center trajectories.

[0117] For this purpose, a cutting and connecting step can be employed. In the cutting step, the ends of all lane center trajectories are considered to determine whether their positions fall within the highest value of their corresponding orientation density map S(D). That is, it is determined whether the position belongs to an area of ​​a specified size (e.g., a square or circular area, where the corresponding orientation density map value is within a predetermined portion (e.g., 25%) of the highest value). If this is not the case, the corresponding ends are cut off, see [reference needed]. Figure 11 The image in the middle.

[0118] To have a directed graph representation, the centers of adjacent lanes can then be connected. Figure 12 Further details of the connection of the extracted lane center trajectories are shown. Here, the end coordinates c of (lane center trajectories A, B, C) are... A c B and c C They are not connected, meaning that not any other lane center trajectory has the same coordinates. First, you could try connecting lane center trajectory A to another lane.

[0119] Therefore, starting from position c A Perform pre-extraction (backward portion). In this case, the trajectory continues until c. B Located in an adjacent region, as described above.

[0120] Therefore, the updated lane center A connects to B, and thus c A =c B Due to this update, c BB is connected, and no action needs to be taken on B. However, the lane center trajectory C remains undecided. The process is similar to that of A, so it will be connected to A. Here, it is the pre-extracted forward portion of C. The result is shown in the right figure. The connection is stopped due to another stopping condition (a seemingly reasonable lane center is within close range).

[0121] In other words, in the connection part, the pre-extracted portion can be used at the end of each unconnected lane center. However, the connection operation can be performed on the density map S(D) without distinguishing the orientation of the density map. Furthermore, the process stops near what appears to be a reasonable lane center. A nearby lane center is reasonable if the orientation of the lane center is opposite to the orientation of the considered trajectory (which will be connected). Figure 11 and Figure 12 The right figure shows the result of the connection processing.

[0122] Those skilled in the art will understand that the above-described embodiments for extracting the trajectory of the lane center are deterministic methods. These embodiments advantageously avoid the application of machine tilt algorithms, which would otherwise require extensive training based on ground realities that may not be available for any traffic environment or any geographical area. However, appropriately trained machine learning algorithms can also be applied to extract the trajectory of the lane center.

[0123] Each lane center can be given as a list of coordinates ordered with respect to the direction of travel. Due to the extraction method described above, the coordinates are concatenated and can contain redundant coordinates for the representation of lane centers, for example, in the case of straight lanes. Redundant coordinates can be removed, for example, by the Ramer-Douglas-Peucker algorithm [9]: Ramer et al. “An iterative procedure for the polygonal approximation of plane curves”, Computer Graphics and Image Processing, 1(3): 244-256, 1972, and

[10] : Douglas et al. “Algorithms for the reduction of the number of points required to represent a digitized line or its caricature”, International Journal of Geographic Information and Geographic Visualization, 10(2): 112–122, 1973. This can also be used to smooth the extracted lane center trajectories. In terms of the directed graph representation of the map, the lane center trajectory, i.e., the connection of two coordinates, can be regarded as an edge, and the coordinates themselves and the connection of the lane center trajectory can be regarded as nodes (e.g., such as Figure 7 (As shown in the diagram of a fork or merge).

[0124] The lane center trajectories extracted from the four-directional trajectory maps can then be reprojected onto the map and world coordinates and collected in a single map. This allows the clustered trajectories of the trajectory centers to be represented in a directed graph representation (i.e., a structured dataset) that includes the corresponding driving directions (HD map). The directed graph representation can also include one or more representations of vehicle density, lane bifurcation, lane merging, and lane boundaries.

[0125] Such HD maps can therefore be easily generated and continuously updated by cloud server 20. The HD maps (and associated collected metadata, see below) can be downloaded to fleet 20 and can be advantageously used for improved ADAS functions. That is, the HD maps can be used to define the framework in which vehicles 20 can move, and can also serve as a backup solution when the on-site ADAS system malfunctions or its cameras are obstructed.

[0126] In a preferred embodiment, the determination of the aforementioned aggregated trajectories can be extended to include aggregated pedestrian trajectories as well. That is, sensor-based data can also be used to determine multiple individual pedestrian trajectories (third trajectories), and these multiple individual pedestrian trajectories can be aggregated to determine aggregated pedestrian trajectories that can also be included in a grid map representation of the map. In other words, the map can therefore also include pedestrian-related information such as sidewalks, zebra crossings, etc. Here, pedestrian detection and the aggregation of individual trajectories can be performed using the aforementioned sensor-based detection and the aggregation of individual trajectories illustrated in the example for vehicular traffic.

[0127] Therefore, the generated map is more reliable than a regular map because it better reflects the current traffic conditions, which may also include specific and accurate information about pedestrian movement and lane crossings.

[0128] Figure 13 A method for determining vehicle category-specific cluster trajectories according to another embodiment is shown. Figure 13 Step S310: Acquire sensor-based data and positioning data (as described above for...) Figure 5 (as explained in step S110).

[0129] according to Figure 13 Step S330 can further distinguish the various trajectories of vehicles based on multiple vehicle categories (as explained above). Figure 5 (as determined in step S130).

[0130] That is, multiple vehicle categories can be considered, such as a first vehicle category concerning automobiles, a second vehicle category concerning buses, a third vehicle category concerning trucks, a fourth vehicle category concerning motorcycles, and a fifth vehicle category concerning bicycles. Those skilled in the art will understand that this is a non-limiting example of such object classes, and other more or less specific categories can also be considered, such as a first vehicle category concerning four-wheeled vehicles and a second vehicle category concerning two-wheeled vehicles.

[0131] As described above, using the acquired sensor data, multiple different vehicles can thus be detected and tracked over time to determine a separate trajectory (a vehicle category-specific second trajectory) for each of the multiple vehicle categories. Furthermore, using positioning data (e.g., using GPS) in the GCS, the relative positions detected by the vehicle's radar and / or lidar, as well as the vehicle tracking, can also be converted into the GCS, so that individual vehicle trajectories can also be provided in the GCS. Therefore, the vehicle category-specific second trajectory can generally also be considered as the path or trajectory (set of spatial coordinates) followed by the tracked vehicle in the GCS over time.

[0132] according to Figure 13 In step S350, the method can also aggregate each of the individual vehicle category-specific trajectories to determine multiple vehicle category-specific aggregated trajectories in the grid cell map representation of the map (HD map).

[0133] Therefore, the map can accumulate presence, direction of travel, and observed speed, which can be differentiated for different vehicle categories. Based on this, specific lanes, such as bus lanes, can be detected, but bypass lanes or other special lanes that can only be used by specific types of vehicles can also be detected.

[0134] Figure 13 The preferred implementation uses radar-based sensor data to determine individual trajectories, avoiding the cost of using lidar sensors to generate maps. In other words, this is a low-cost variant that creates, maintains, and updates such maps based on readily available information from existing ADAS vehicles.

[0135] Furthermore, when determining each trajectory, metadata associated with each trajectory can also be saved, such as speed data (e.g., based on detected Doppler values) stored in association with each trajectory, distance to the vehicle, classification results, and the date and time of trajectory detection. That is, the information of the determined second trajectory is preferably stored together with metadata about the detected vehicle category and / or pedestrian type.

[0136] The stored metadata may also be included in the aggregated trajectory, which may include metadata related to the creation and / or use of the aggregated trajectory.

[0137] This metadata can be advantageously used to aggregate individual second trajectories determined only within approximately the last hour (i.e., within a specific timeframe). Using this metadata, maps can be constructed based on convoy detection and tracking information over the last, for example, 5 hours or 2 days (i.e., a specific timeframe).

[0138] This further allows for the generation of highly accurate aggregated trajectories and corresponding metadata, such as average speeds that better reflect current traffic conditions. Those skilled in the art will understand that this is useful in scenarios where, for example, the legal speed limits on a road may have changed without updating the corresponding map data, or a construction zone may suddenly render mapped lane markings invalid. The metadata may also include confidence values ​​based on the total number of aggregated individual trajectories (second trajectories).

[0139] Therefore, the resulting map can provide accurate and up-to-date metadata, such as empirical speeds (i.e., speeds actually observed rather than legally permitted speeds). Current changes in empirical speeds are valuable information for map change detection (e.g., construction sites, temporary traffic restrictions, etc.).

[0140] Only after a critical number of "consensus" has been reached, that is, only after a specific minimum number of individual trajectories (overall and / or per class) have been observed for each grid cell of the map, can the aggregated trajectories and the corresponding updated map be made public, for example, by a cloud server.

[0141] Figure 14 A method implemented by a vehicle according to another embodiment is shown. Vehicle 10 has a communication interface (such as...) Figure 4 The vehicle 10 communicates with the cloud server 20 via the I / F 310 in the communication interface. Through this interface, the vehicle 10 can receive a map (HD map) generated according to one of the above embodiments (step S410). The vehicle also has a control unit (e.g., Figure 4 The processor 320, or Figure 2 The vehicle 10 has a driver assistance system 200. The control unit makes advanced driving and safety decisions (e.g., ADAS and / or AD decisions) based on a received map, which includes aggregated trajectories and / or metadata as described above (step S430). Therefore, the vehicle 10 is able to access not only its own perception information extracted from the equipped sensors, but also aggregated trajectories and / or metadata from an aggregation service provided by a cloud server, and thus is able to make improved ADAS decisions.

[0142] For example, the aggregated trajectory and metadata of the received map include updated average speed information about driving in the lane and how recent that information is, which can be used to perform effective route planning.

[0143] According to a preferred embodiment, the control unit can be configured to use only aggregated trajectories in the map that have been aggregated using a minimum number of individual trajectories and / or that have been aggregated using a minimum number of trajectories determined within a specific past time period. In particular, as described above, the aggregated trajectory can carry metadata about how many individual trajectories were used to generate (and verify) the aggregated trajectory. The aggregated trajectory can also carry metadata about how old the most recent trajectory was when it was aggregated into this aggregated trajectory. Therefore, the control unit of the ADAS system can now determine the "quality" of the aggregated trajectory by using only those trajectories that: a) have received a sufficient total number of individual trajectories (e.g., at least 100 individual trajectories); and / or b) have received a sufficient number of recent trajectories (e.g., at least 10 trajectories within the last 10 minutes). Those skilled in the art will understand that the specific examples of the minimum number of individual trajectories and the minimum number of trajectories within the last specific time period are non-limiting examples.

[0144] This method ensures that the ADAS system's control unit can verify the correctness and accuracy of the adopted trajectory before using these aggregated trajectories for ADAS decision-making. In other words, the control unit can also be configured to output control commands to the vehicle to follow one of the aggregated trajectories extracted from the received map, especially when the vehicle's cameras are obstructed.

[0145] More specifically, ADAS vehicle 10 can be equipped with existing L2 autonomous driving capabilities and can therefore actively follow lane markings by observing them with its own perception sensors (e.g., a camera system), and / or thus actively control its longitudinal distance to any vehicle ahead (e.g., by deploying radar for distance measurement). A typical ADAS vehicle will cease operation and warn the driver to take over control of the vehicle if and only if the perception software fails to observe its surroundings (e.g., due to camera obstruction or direct sunlight). This interruption can occur spontaneously without driver warning. Because this system requires immediate human driver intervention and therefore constant driver attention, it can only qualify as an L2 autonomous driving system.

[0146] As additional information (aggregation trajectory and / or meta-information) from the cloud aggregation service of cloud server 20 becomes available, and preferably as the "correctness" and "accuracy" are confirmed as outlined above, the control unit of the ADAS system can now implement different methods to handle this "obstruction" situation of the perception sensors. Instead of directly disabling the ADAS function and expecting the driver to take immediate control, the ADAS system can therefore revert to following the aggregation trajectory on the map it receives from the cloud service (cloud server).

[0147] The control unit preferably applies this rollback only under the following conditions: as stated above, a) perception of other road users (e.g., based on radar sensors) is still in effect, thus ensuring that there are no actual obstacles on the road ahead; and b) the convergence trajectory of the map used is up-to-date and accurate.

[0148] In another preferred embodiment of making ADAS decisions based on the received map, the control unit may be further configured to output control commands to the vehicle 10 to follow the convergence trajectory when the convergence trajectory is accurate.

[0149] That is, data related to the aggregation trajectories and / or metadata received from the cloud aggregation service (cloud server 20) provides the ADAS system with "look-ahead" information to achieve "trajectory following". As long as there is enough "look-ahead" information available, and therefore enough aggregation trajectories that meet the above conditions can be considered "up-to-date" and / or "accurate", the control unit can continue to operate normally by following these trajectories, similar to the traditional "platooning" concept.

[0150] At some point, there will be a situation where no more aggregated tracks can be used because a) other ADAS vehicles have not previously observed more tracks, so the aggregated track service cannot provide any information, or b) the provided tracks are no longer qualified as "up-to-date" and "accurate" because too little or too old data has been aggregated. In this case, the ADAS system's control unit can notify the driver of an alert that it will shut down its functions, similar to the situation without such additional track information. However, in general, the lead time for driver takeover will be longer because the ADAS system can "advance" and "pre-determine" when it will run out of enough information to maintain functionality.

[0151] In another preferred embodiment of making ADAS decisions based on the received map, the control unit may be further configured to determine a recommended cornering speed based on the received map, particularly the shape of the converged trajectory and the metadata contained therein, such as average speed information.

[0152] In other words, by applying the "look-ahead" provided by the aggregated trajectory adopted from the aggregation service, the ADAS vehicle's control unit can calculate the appropriate speed to reach the next curve. This recommended speed can be calculated as in classic speed recommendation systems (e.g., based on road curvature), but without requiring perception software to actually perceive the road geometry ahead. This allows for smoother speed adjustments and more time for the vehicle to decelerate. Depending on the difference between the actual and recommended speeds, the system can even continue operating instead of warning the driver and shutting down. This significantly improves the comfort and perceptual usability of speed recommendation functions, thus broadening the acceptance of such systems.

[0153] In the foregoing description, various aspects have been described with reference to multiple embodiments. Therefore, the specification should be considered illustrative rather than restrictive. Similarly, the figures shown in the accompanying drawings, highlighting the functionality and advantages of the embodiments, are presented for illustrative purposes only. The architecture of the embodiments is flexible and configurable enough that it can be used in ways other than those shown in the accompanying drawings.

[0154] In exemplary embodiments, the software implementations presented herein can be provided as computer programs or software, such as one or more programs having instructions or sequences of instructions, which are included or stored in an article of manufacture such as a machine-accessible or machine-readable medium, an instruction store, or a computer-readable storage device, each of which may be non-transitory. Programs or instructions on a non-transitory machine-accessible medium, machine-readable medium, instruction store, or computer-readable storage device can be used to program a computer system or other electronic equipment. Machine or computer-readable media, instruction stores, and storage devices may include, but are not limited to, floppy disks, optical disks, and magneto-optical disks, or other types of media / machine-readable media / instruction stores / storage devices suitable for storing or transmitting electronic instructions. The techniques described herein are not limited to any particular software configuration. They can be found in any computing or processing environment. As used herein, the terms “computer-readable medium,” “machine-accessible medium,” “machine-readable medium,” “instruction store,” and “computer-readable storage device” should include any medium capable of storing, encoding, or transmitting instructions or sequences of instructions for execution by a machine, computer, or computer processor, and enabling the machine / computer / computer processor to perform any of the methods described herein. Furthermore, in this art, software is often referred to as taking an action or causing a result in one form or another (e.g., program, procedure, process, application, module, unit, logic, etc.). Such expressions are merely shorthand ways of describing how a processing system executes software to cause the processor to perform actions to produce a result.

[0155] Some implementations can also be achieved by preparing application-specific integrated circuits, field-programmable gate arrays, or by using appropriate networks to interconnect conventional component circuits.

[0156] Some implementations include a computer program product. A computer program product may be a storage medium or multiple storage media, one or more instruction stores, or one or more storage devices having instructions stored thereon or therein, which can be used to control or cause a computer or computer processor to execute any program of the exemplary implementations described herein. The storage medium / instruction store / storage device may include (e.g., but not limited to) optical discs, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory, flash memory cards, magnetic cards, optical cards, nanosystems, molecular memory integrated circuits, RAID, remote data storage / archiving / storage, and / or any other type of device suitable for storing instructions and / or data.

[0157] Some implementations stored on any of a computer-readable medium or multiple media, one or more instruction stores, or one or more storage devices include hardware for controlling the system and software for enabling the system or microprocessor to interact with a human user or other mechanisms using the results of the embodiments described herein. Such software may include, but is not limited to, device drivers, operating systems, and user applications. Finally, such computer-readable media or storage devices also include software for performing the exemplary aspects described above.

[0158] Included in the system's programming and / or software are software modules used to implement the processes described herein. In some example embodiments herein, the modules comprise software, although in other example embodiments herein, the modules comprise hardware or a combination of hardware and software.

[0159] While various embodiments of this disclosure have been described above, it should be understood that they are presented as examples and not as limitations. It will be apparent to those skilled in the art that various changes in form and detail can be made therein. Therefore, the exemplary embodiments described above are not restrictive.

[0160] Furthermore, the purpose of this abstract is to enable patent offices and the public, especially scientists, engineers, and practitioners in the field who are unfamiliar with patent or legal terminology or wording, to quickly determine the substance and nature of the technical disclosure of this application from a cursory examination. The abstract is not intended to limit the scope of the embodiments presented herein in any way. It should also be understood that any process recited in the claims need not be performed in the order presented.

[0161] Although this specification contains numerous specific details of implementation, these details should not be construed as limiting the scope of the claims, but rather as descriptions of specific features of particular embodiments described herein. Certain features described herein in the context of a single embodiment may also be implemented in combination in that single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments. Furthermore, although features may be described above as functioning in certain combinations and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in certain circumstances, and the claimed combination may involve sub-combinations or variations thereof.

[0162] In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of the various components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0163] Some illustrative embodiments have now been described, and it is clear that the content given above by way of example is illustrative and not restrictive. In particular, although many of the examples given herein involve specific combinations of apparatus or software elements, these elements can be combined in other ways to achieve the same purpose. Actions, elements, and features discussed in connection with only one embodiment are not intended to be excluded from similar roles in other embodiments or multiple embodiments.

[0164] The apparatuses, devices, and units described herein may be implemented in other specific forms without departing from their characteristics. The above embodiments are illustrative and not limiting of the systems and methods described. Therefore, the scope of the apparatuses described herein is indicated by the appended claims rather than the foregoing description, and modifications falling within the meaning and scope of equivalents of the claims are included therein.

Claims

1. A computer-based method for generating maps, the computer-based method comprising the following steps: - Acquire sensor-based data from at least one vehicle's sensor unit and positioning data associated with the positioning unit of the at least one vehicle; - The positioning data is used to determine a first trajectory of the at least one vehicle, and the sensor-based data is used to determine multiple second trajectories of other vehicles; - Aggregate the first trajectory and / or the plurality of second trajectories to generate an aggregated trajectory in a grid map representation of the map; as well as - Extract the lane center trajectory from the aggregated trajectory of the map. The computer implementation method further includes classifying the first trajectory and the plurality of second trajectories according to a predetermined set of different driving directions, and generating a plurality of driving direction-specific clustered trajectories, wherein the clustered trajectories are related to the density map. D The correlation is that the specific clustering trajectory in the driving direction is related to the driving direction density map. D d Relatedly, the density map and the driving direction density map are generated using data collected from the first trajectory and / or the plurality of second trajectories, wherein the density map is based on the processed density map. S(D) and / or processed driving direction density map S(D d ) To perform the extraction of the trajectory at the center of the lane, the processed density map S(D) and the processed driving direction density map S(D d ) By analyzing the density map D and the driving direction density map D d The function is used to determine the corresponding local density maximum value in the direction perpendicular to the aggregation trajectory and the specific aggregation trajectory of the driving direction, and wherein the extraction of the trajectory of the lane center is performed by tracing the ordered path of the local density maximum value.

2. The computer implementation method according to claim 1, wherein, The extraction of the trajectory at the center of the lane is performed by selecting a cluster trajectory value that is greater than or equal to a predetermined threshold.

3. The computer implementation method according to claim 1, wherein, The trajectory of the lane center is post-processed to connect the ends of the extracted trajectory of the lane center.

4. The computer implementation method according to claim 1, further comprising: - The clustering trajectory is represented in a directed graph representation of an HD map that includes the direction of travel.

5. The computer implementation method according to claim 4, wherein, The directed graph representation also includes one or more representations of vehicle density, lane bifurcation, lane merging, and lane boundaries.

6. The computer implementation method according to claim 1, further comprising: - Use the sensor-based data to determine multiple third trajectories for pedestrians; as well as - Aggregate the multiple third trajectories to generate aggregated pedestrian trajectories in the grid map representation of the map.

7. The computer implementation method according to claim 4, wherein: - Multiple second trajectories for other vehicles are determined by differentiating other vehicles based on vehicle category and identifying a set of vehicle category-specific second trajectories; and The computer implementation method further includes: - Aggregate multiple vehicle category-specific second trajectories to generate vehicle category-specific aggregated trajectories in the grid map representation of the HD map.

8. The computer implementation method according to claim 1, further comprising: - Store the determined second trajectory together with the metadata associated with the second trajectory and / or the aggregated trajectory.

9. The computer implementation method according to claim 1, further comprising: - Store the determined second trajectory along with the metadata related to the corresponding detection of the category.

10. The computer implementation method according to claim 4, wherein, The grid map of the HD map represents the average speed information of each grid cell of the HD map.

11. The computer implementation method according to claim 1, wherein, The sensor-based data is based on radar sensor data and / or lidar sensor data.

12. The computer implementation method according to claim 1, wherein, The location data is based on GPG data.

13. A cloud server adapted to perform a computer-implemented method according to any one of claims 1 to 12.

14. A vehicle, the vehicle comprising: - A communication interface configured to receive a map from a cloud server as claimed in claim 13; as well as - Control unit, which is configured to make advanced driving and safety decisions based on the received map.

15. The vehicle according to claim 14, in, The control unit is configured to use only aggregated trajectories in the map that have been aggregated using a minimum number of individual trajectories and / or aggregated trajectories that have been aggregated using a minimum number of trajectories determined within a specific amount of time in the past.

16. The vehicle according to claim 14 or 15, in, The control unit is configured to output control commands to the vehicle to follow one of the clustering trajectories from the received map.

17. The vehicle according to claim 16, in, The control unit is also configured to output control commands to the vehicle to follow one of the convergence trajectories, provided that the convergence trajectory is accurate.

18. The vehicle according to claim 14, in, The control unit is also configured to determine a cornering speed recommendation based on the received map.

19. A system comprising a cloud server according to claim 13 and a plurality of vehicles according to any one of claims 14 to 18.

Images