Method for determining a road user's forward reachable set

Abstract

The invention relates to a method for determining a forward reachable set of a road user, comprising the following steps: Recording an observation area of ​​a road traffic infrastructure over an observation period using infrastructure environment sensors, Determining movement assumptions of different types of road users located within the observation area over the observation period, based on the recording, Determining a Forward Reachable Set of a road user currently located within the observation area of ​​the road traffic infrastructure based on the determined movement assumptions of the road user type corresponding to the current road user. The invention further relates to a system for determining a forward reachable set of a road user, a computer program and a machine-readable storage medium.

Description

[0001] The invention relates to a method and a system for determining a forward reachable set of a road user, a computer program and a machine-readable storage medium. State of the art

[0002] The publication WO 2022 / 144559 A1 discloses a method for autonomous motion control and motion planning of a vehicle.

[0003] Disclosure CN 111788532 A discloses a method for recognizing a traffic scenario.

[0004] The patent application DE 10 2021 209 681 A1 discloses a concept for supporting a motor vehicle through infrastructure. Disclosure of the invention

[0005] The object underlying the invention is to provide a concept for determining a road user's Forward Reachable Set.

[0006] This problem is solved by means of the respective subject matter of the independent claims. Advantageous embodiments of the invention are the subject matter of dependent claims.

[0007] Following an initial assessment, a procedure for determining a road user's Forward Reachable Set is provided, comprising the following steps: Recording an observation area of ​​a road traffic infrastructure over an observation period using infrastructure environment sensors, Determining movement assumptions of different types of road users located within the observation area over the observation period, based on the recording, Determining a Forward Reachable Set of a road user currently located within the observation area of ​​the road traffic infrastructure based on the determined movement assumptions of the road user type corresponding to the current road user.

[0008] A second aspect involves providing a system for determining a road user's Forward Reachable Set, including: an infrastructure environment sensor system which is set up to record an observation area of ​​a road traffic infrastructure over an observation period, an investigative facility which is set up to determine movement assumptions of different types of road users located within the observation area over the observation period, based on the recording, wherein the investigative device is set up to determine a Forward Reachable Set of a road user currently located within the observation area of ​​the road traffic infrastructure based on the movement assumptions determined for the road user type corresponding to the current road user.

[0009] According to a third aspect, a computer program is provided, comprising instructions which, when the computer program is executed by the system according to the second aspect, cause it to perform a procedure according to the first aspect.

[0010] According to a fourth aspect, a machine-readable storage medium is provided on which the computer program is stored according to the third aspect.

[0011] The invention is based on the understanding and includes the fact that the above problem is solved by capturing an observation area of ​​road traffic infrastructure using infrastructure environment sensors before determining a forward reachable set of a current road user over an observation period. Infrastructure environment sensors are external environmental sensors, for example, a camera that observes road traffic. Intelligent infrastructure environment sensors can automatically evaluate these images and, for example, generate information about object movements. The infrastructure environment sensors are, for example, intelligent infrastructure environment sensors. These sensors are, for example, configured to evaluate the environmental data based on the captured data, representing what has been captured, and to determine or generate information about object movements.Based on this observation, movement assumptions of different types of road users located within the observation area over the observation period are determined. This serves as the basis for determining the forward reachable set of the current road user.

[0012] Thus, infrastructure information and movement assumptions are available for determining the forward reachable set for the current road user, and these are gathered over a longer period, the observation period. This allows the forward reachable set to be determined efficiently.

[0013] Including the infrastructure to determine the Forward Reachable Set is particularly advantageous because the conditions of the roads and signage usually remain unchanged over the observation period, and movements of road users can be considered over the observation period to characterize the reasonably assumed movement of dynamic objects, in this case the road users, who were in the observation area during the observation period.

[0014] According to the concept described here, historical data, i.e., data obtained from the recording over the observation period, are used to calculate or determine the Forward Reachable Set for a road user who is currently within the observation area of ​​the road traffic infrastructure.

[0015] This results in the particular technical advantage of providing a concept for determining a road user's Forward Reachable Set.

[0016] A forward reachable set, as described here, specifically describes all sets that can be reached from an initial set. A road user's forward reachable set thus describes all sets that can be reached from the road user's initial set. The initial set can also be referred to as an initial state or a starting state. The sets that can be reached from this initial set can also be referred to as target states or target set(s).

[0017] A partially automated vehicle can demonstrably drive without collisions using the concept of reachable sets, and in particular forward reachable sets, provided the underlying motion assumptions are valid. Because the forward reachable set can be efficiently determined according to the concept described here, it can be ensured that a partially automated vehicle can drive without collisions with a high probability.

[0018] At least partially automated systems include the following: assisted, semi-automated, highly automated, fully automated, and autonomous. Assisted corresponds to automation level 1 according to the definition of the Federal Highway Research Institute (BASt). Semi-automated corresponds to automation level 2 according to the BASt definition. Highly automated corresponds to automation level 3 according to the BASt definition. Fully automated corresponds to automation level 4 according to the BASt definition. Autonomous corresponds to automation level 5 according to SAE (J3016), where SAE stands for "Society of Automotive Engineers".

[0019] In one embodiment of the method, it is provided that the motion assumptions for different situations are determined, whereby the current situation of the current road user is determined, and based on the current situation, one or more of the determined motion assumptions are selected, and the Forward Reachable Set is determined based on the selected motion assumption(s).

[0020] This results, for example, in the technical advantage that the most suitable motion assumptions can be used for the current situation to calculate the Forward Reachable Set that best corresponds to the current situation.

[0021] In one embodiment of the method, the situation is characterized by one or more of the following situation parameters: traffic time, in particular peak traffic time, off-peak traffic time, low traffic time or night traffic time, season, in particular spring, summer, autumn or winter, weather, holiday time, no holiday time, time of day, market day, no market day, occurrence or non-occurrence of a traffic-relevant event, in particular a sporting event or concert.

[0022] This results, for example, in the technical advantage that particularly suitable situation parameters can be used to characterize the situation.

[0023] In one embodiment of the method, it is provided that determining the motion assumptions includes determining sets of parameters, each of which represents a motion assumption.

[0024] This results, for example, in the technical advantage that the assumptions about motion can be efficiently represented or characterized.

[0025] In one embodiment of the method, the parameter sets each include one or more of the following parameters: road user type, specification of the area of ​​the observation range for which the parameter set is valid, specification of a restriction of the Forward Reachable Set to be determined, maximum longitudinal speed, maximum lateral speed, maximum longitudinal acceleration, maximum lateral acceleration, maximum longitudinal deceleration, maximum lateral deceleration, maximum change in a direction of movement per unit of time.

[0026] This results, for example, in the technical advantage that the parameter sets include particularly suitable parameters, so that the motion assumptions can be represented accordingly efficiently.

[0027] In one embodiment of the method, it is provided that the Forward Reachable Set is determined on the infrastructure side and then wirelessly transmitted to a motor vehicle that is at least partially automated.

[0028] This results, for example, in the technical advantage that the at least partially automated vehicle can be driven efficiently, at least partially, based on the Forward Reachable Set.

[0029] In one embodiment of the method, it is provided that the motion assumptions are determined on the infrastructure side, wherein at least one of the motion assumptions determined on the infrastructure side is wirelessly sent to a motor vehicle that is at least partially automated, wherein the Forward Reachable Set is determined on the motor vehicle side based on the at least one motion assumption.

[0030] This results, for example, in the technical advantage that the vehicle itself can efficiently determine the Forward Reachable Set.

[0031] Statements made in connection with the system apply analogously to the process and vice versa. This means, in particular, that the technical functionalities and technical characteristics of the process result analogously from corresponding technical functionalities and characteristics of the system, and vice versa.

[0032] The system is set up, for example, to execute all steps of the procedure.

[0033] The process is carried out, for example, using the system.

[0034] The method is, for example, a computer-implemented method.

[0035] The system is, for example, programmed to execute the computer program.

[0036] An infrastructure environmental sensor system comprises, for example, one or more environmental sensors. These environmental sensors are spatially distributed within the infrastructure, i.e., located within the observation area and / or outside the observation area, in which case the corresponding environmental sensors can cover the observation area.

[0037] An environmental sensor is, for example, one of the following environmental sensors: radar sensor, lidar sensor, ultrasonic sensor, image sensor, in particular the image sensor of a camera, infrared sensor and magnetic field sensor.

[0038] An observation area as defined in the description includes, for example, one or more of the following road traffic infrastructure elements: junction, intersection, T-junction, roundabout, crossing, pedestrian crossing, traffic signal system, motorway junction, motorway entrance, motorway exit, motorway junction, level crossing.

[0039] An observation period as defined in the description is at least one month, in particular several months, in particular at least one year, in particular at least one season.

[0040] A road user type or road user within the meaning of the description is, for example, one of the following road user types or road users: motor vehicle, motorcycle, bicycle, pedestrian, truck, passenger car, moped, scooter.

[0041] The invention is explained in more detail below with reference to preferred embodiments. These include: Fig. 1. A flowchart of a procedure for determining a road user's forward reachable set, Fig. 2. A system for determining a road user's forward reachable set, Fig. 3 a machine-readable storage medium, Fig.4 a schematic representation of a road user's forward reachable set, Fig. 5 a block diagram and Fig. 6 an observation area of ​​a road traffic infrastructure.

[0042] Fig. Figure 1 shows a process for determining a road user's Forward Reachable Set, comprising the following steps: Recording 101 of an observation area of ​​a road traffic infrastructure over an observation period using infrastructure environment sensors, Determine 103 movement assumptions of different types of road users located within the observation area over the observation period, based on the recording, Determine 105 a Forward Reachable Set of a road user currently located within the observation area of ​​the road traffic infrastructure based on the movement assumptions determined for the road user type corresponding to the current road user.

[0043] Fig. Figure 2 shows a System 201 for determining a road user's Forward Reachable Set, comprising: an infrastructure environment sensor system 203, which is set up to record an observation area of ​​a road traffic infrastructure over an observation period, an investigative device 205, which is set up to determine movement assumptions of various types of road users located within the observation area over the observation period based on the recording, wherein the investigation device 205 is set up to determine a forward reachable set of a road user currently located within the observation area of ​​the road traffic infrastructure based on the movement assumptions determined of the road user type corresponding to the current road user.

[0044] For example, system 201 includes a communication device which is configured, for example, to wirelessly transmit the determined forward reachable set to a motor vehicle with at least partial automation. For example, the communication device is configured to wirelessly transmit at least one, in particular several, in particular all, of the movement assumptions determined by the infrastructure to a motor vehicle with at least partial automation.

[0045] The investigative unit is, for example, set up to carry out the exemplary steps mentioned in connection with the procedure.

[0046] Fig. Figure 3 shows a machine-readable storage medium 301 on which a computer program 303 is stored. The computer program 303 comprises instructions that are executed by the system 201 when the computer program 303 is executed. Fig. 2. cause this to be carried out in accordance with the first aspect.

[0047] Fig. Figure 4 schematically shows a forward reachable set 401 of a road user who is in an initial state 403. The forward reachable set 401 comprises a target set of possible trajectories 405 of the road user starting from the initial state 403.

[0048] Fig. Figure 5 shows a block diagram 501, which illustrates the concept described here.

[0049] According to functional block 503, sensing by the infrastructure is provided. In other words, according to functional block 503, the step of recording an observation area of ​​a road traffic infrastructure over an observation period is carried out using infrastructure environment sensors.

[0050] The data collected is processed on the infrastructure side according to function block 505. This means that, based on the data collected, movement assumptions are determined for various types of road users who were within the observation area during the observation period.

[0051] For example, according to function block 505, it may be provided that a Forward Reachable Set is determined on the infrastructure side for a road user currently located within the observation area of ​​the road traffic infrastructure, whereby this determination is carried out based on the movement assumptions determined for the road user type corresponding to the current road user.

[0052] According to Function Block 507, the determined Forward Reachable Set is to be sent to a vehicle with at least partial automation that is currently approaching or already within the observation area. For example, Function Block 507 may stipulate the following infrastructure step: sending information about objects, in particular movement possibilities and / or conditions of a region, i.e., the observation area.

[0053] For example, according to function block 507, it may be provided that the movement assumptions determined by the infrastructure are sent to a motor vehicle that is at least partially automated and is currently approaching or within the observation area.

[0054] According to functional block 509, it is provided that the data or information sent by the infrastructure, i.e., for example, the determined movement assumptions and / or the determined Forward Reachable Set, are received by the at least partially automated vehicle.

[0055] On the vehicle side, the information or data is processed, for example, by an ADS. ADS stands for "Automated Driving System".

[0056] According to Function Block 511, the received data or information is processed by the vehicle, in particular by the ADS, to determine, for example, a Forward Reachable Set for a road user currently within the observation area, whereby this determination is based on at least one movement assumption. For example, according to Function Block 511, it may be provided that the Forward Reachable Set transmitted by the infrastructure is used by the vehicle to enable at least semi-automated driving of the vehicle by the ADS.

[0057] According to function block 513, for example, the determined Forward Reachable Set (FRS), whether determined by the vehicle or the infrastructure, is used by the ADS to enable at least semi-automated driving of the vehicle, in accordance with a safety guarantee. For example, function block 513 may provide for the ADS to determine and output control signals for at least semi-automated lateral and / or longitudinal control of the vehicle.

[0058] Fig. Figure 6 shows a road traffic infrastructure 601 comprising an observation area which includes a pedestrian crossing 603 and a road area which is located to the right of the pedestrian crossing in relation to the paper plane.

[0059] A camera 605 is located in the immediate vicinity of the pedestrian crossing 603 and records the crossing and the road leading to it. For example, several cameras may be used to ensure sufficient coverage of the observation area.

[0060] The camera 605, or multiple cameras 605, can monitor this observation area, in particular the pedestrian crossing 603, over an observation period, for example, several months or at least one year, and the movements of various road users can be determined over this observation period. For example, the movements of pedestrians crossing the pedestrian crossing 603 can be determined. These movements, or trajectories, are in Fig.6 is symbolically marked with reference symbol 607. Furthermore, this observation allows the determination of the trajectories or movements of motor vehicles approaching and crossing pedestrian crossing 603 on roadway 609. Exemplary trajectories are marked with reference symbol 611.

[0061] This provides data showing the typical movements of different road users within the observation area.

[0062] How Fig. Figure 6 shows, for example, that pedestrians properly cross the roadway 609 via the pedestrian crossing 603 and do not deviate from it.

[0063] Based on this data, movement assumptions for different types of road users can be determined. Based on this information, i.e., based on these movement assumptions, a forward reachable set can then be determined for a road user currently within the observation area of ​​road traffic infrastructure 601.

[0064] For example, camera 605 can detect a pedestrian who is currently about to cross pedestrian crossing 603. Based on the previously determined movement assumptions, a forward reachable set can then be determined for this pedestrian by the infrastructure. This determined forward reachable set can be wirelessly transmitted to a vehicle with at least semi-automated driving capabilities, so that the vehicle can be guided, at least semi-automatically, based on this forward reachable set as it approaches pedestrian crossing 603.

[0065] In summary, the concept described here specifically aims to use infrastructure information, i.e., the data obtained from the recording over the observation period, to efficiently determine a Forward Reachable Set for a road user currently within the observation area of ​​the road traffic infrastructure.

[0066] Including the infrastructure offers advantages here, as the road and signage conditions remain unchanged, and the movements of objects can be observed over longer periods to characterize the reasonably expected movement (motion assumption) of dynamic objects (road users). Furthermore, the movements of an ADS (Automated Dynamic System) expected in the current environment can be explicitly restricted, for example, by using forward reachable sets provided by the infrastructure. A particular advantage is the ability to adapt the forward reachable set to specific local conditions.

[0067] According to the concept described here, it is intended, for example, that the Forward Reachable Set is determined on the infrastructure side and / or on the vehicle side. In the latter case, it is specifically intended that the corresponding motion assumptions are sent to the vehicle.

[0068] The following table provides further examples illustrating the concept described here. Features from the table can be used individually or in combination with other embodiments described here. "FRS" stands for "Forward Reachable Set". variant Examples Partitioning the task The infrastructure collects data for various object classes and generates aggregated movement data from it. This data is used by the AV (Automatic Vehicle) to make assumptions about expected movements. Motion models of potential objects behind static occlusions; characterization of entry areas at intersections; characterization of actually used traffic lanes The Forward Reachable Set is calculated in the vehicle. Information about possible movement patterns is included. Infrastructure transmits motion and prediction models validated from long-term observation for Road users for defined areas. Infrastructure classifies regions, and the ADS then applies different specifications depending on the region. For example, structured / unstructured. Conditions of the roads and signage etc. are transferred from the infrastructure. The Forward Reachable Set is calculated in the vehicle. The region information is incorporated in such a way that region-specific parameters are selected. Only the type of region is used. Infrastructure defines the movement possibilities of objects to be considered by security systems and, if necessary, secures them through appropriate measures. The movement possibilities of the objects under consideration are determined by the infrastructure. It is particularly advantageous that the infrastructure can incorporate the signal positions of the immediate surroundings to define the movement possibilities. It is also particularly advantageous that the infrastructure, through traffic lights and variable message signs, can design the boundary conditions in such a way that movement possibilities are strictly adhered to. The vehicle adopts the motion models specified by the infrastructure for the Forward Reachable Set calculation. Or: The vehicle directly adopts Forward Reachable Sets for objects provided by the infrastructure. An ADS is assigned a movement model by the infrastructure and guarantees compliance for a given area. The movement possibilities of the objects under consideration are strictly determined by the infrastructure – by variable message signs and lighting systems. The ADS confirms this. An ADS receives an FRS through the infrastructure. The infrastructure transmits at an intersection with a currently red traffic light for The FRS is calculated in the infrastructure. assigned, which specifically restricts the permitted trajectories of ADHD. Left-turning vehicles, but without restrictions for right-turning vehicles or straight-ahead vehicles, send a corresponding "L"-shaped FRS to the ADS. and transmitted via V2X to an ADS. Information available in the infrastructure about the status of signaling systems and (planned or predicted) movements of other objects is incorporated into the FRS calculation.

[0069] The following are exemplary parameters that may be used to describe or characterize a motion assumption or assumptions.

[0070] Parameter set of movement assumptions (example) for a region (sub-area of ​​infrastructure monitoring). The movement assumptions specify the parameters obtained from long-term observations, namely the reasonably foreseeable worst-case assumptions. Type of road user:

[0071] Examples: pedestrians, cars (passenger vehicles), trucks (lorries). Area for which this statement is valid: Specifying the region, e.g., sub-area of ​​the observation area. Fundamental limitations of Forward Reachable Sets: Areas inaccessible from the region, which fundamentally limit forward reachable sets, for example, areas where participants have never been observed. (This is usually due to a physical barrier or traffic rules that are strictly followed at this location. The reason is not detected, only that no road users from a neighboring position enter this area.) Maximum longitudinal speed v_lon_max Maximum lateral speed v_lat_max Maximum longitudinal acceleration a_lon_max Maximum lateral acceleration a_lat_max Maximum longitudinal deceleration b_lon_max Maximum lateral deceleration b_lat_max Maximum heading angle rate h_dot

[0072] Long-term observation: In contrast to short-term observation and ad-hoc assessment of a specific road user, the concept described here utilizes long-term observation of a wide variety of road users. Based on these observations, parameter sets for movement assumptions are created, as exemplified above. This involves, for example, measuring and statistically evaluating the relevant parameters and finally entering the parameters that correspond to the safety-conservative movement assumptions (the reasonably foreseeable worst-case scenarios) into a transferable format.

[0073] A long-term observation, for example, includes a period for which the measurements are intended to be representative and in which a statistically significant number of observations have been made.

[0074] Long-term observations can include, for example, various daily and seasonal fluctuations.

[0075] The parameter sets can be determined for different situations. This can be done, for example, by a programmer entering the relevant time intervals and generating a parameter set for each combination. In the following example, 128 parameter sets are determined. The observation time without prior knowledge is generally at least one year. However, predefined values ​​can also be used as starting values, which can then be refined over time. Traffic times season Weather Other 1 Other2 rush hour Spring precipitation Holidays Market days Off-peak hours Summer No precipitation no holidays no market days Off-peak hours Autumn Nighttime winter

[0076] Using automated methods that are also used for analyses and predictions in traffic management, corresponding parameter sets can be generated, for example.

[0077] This section briefly outlines an example of a machine learning (ML) / artificial intelligence (AI) approach to illustrate a possible implementation. Input data for such a training process includes, for example, historical data such as traffic flow, weather data, and recorded movement patterns, or even the parameters of those recorded movement patterns. Statistical methods or machine learning techniques can then be used to determine the relevant parameter sets for the specific situation.

[0078] For example, the appropriate parameter set can be determined from the current traffic flow and the date and time using the trained model.

[0079] A wide variety of machine learning and algorithmic (ML) methods are available. A safety function benefits from traceability, therefore a so-called decision tree (DT) can be very effectively used here. In this case, the logic is understandable to humans. QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] WO 2022 / 144559 A1

[0002] CN 111788532 A

[0003] DE 10 2021 209 681 A1

[0004]

Claims

[1] Method for determining a road user’s Forward Reachable Set (401), comprising the following steps: Recording (101) an observation area of ​​a road traffic infrastructure (601) over an observation period using infrastructure environment sensors, Determining (103) movement assumptions of different types of road users located within the observation area over the observation period based on the recording, Determining (105) a Forward Reachable Set (401) of a road user currently within the observation area of ​​the road traffic infrastructure based on the movement assumptions determined of the road user type corresponding to the current road user. [2] Method according to claim 1, wherein the motion assumptions for different situations are determined, wherein the current situation of the current road user is determined, wherein one or more of the determined motion assumptions are selected based on the current situation, and wherein the Forward Reachable Set (401) is determined based on the selected motion assumption(s). [3] Method according to claim 2, wherein the situation is characterized by one or more of the following situation parameters: traffic time, in particular peak traffic time, off-peak traffic time, low traffic time or night traffic time, season, in particular spring, summer, autumn or winter, weather, holiday time, no holiday time, time of day, market day, no market day, occurrence or non-occurrence of a traffic-relevant event, in particular a sporting event or concert. [4] Method according to any of the preceding claims, wherein determining the motion assumptions includes determining sets of parameters, each of which represents a motion assumption. [5] Method according to claim 4, wherein the parameter sets each comprise one or more of the following parameters: road user type, indication of the area of ​​the observation range for which the parameter set is valid, specification of a restriction of the Forward Reachable Set (401) to be determined, maximum longitudinal speed, maximum lateral speed, maximum longitudinal acceleration, maximum lateral acceleration, maximum longitudinal deceleration, maximum lateral deceleration, maximum change of a direction of motion per unit time. [6] Method according to one of the preceding claims, wherein the Forward Reachable Set (401) is determined on the infrastructure side, which is wirelessly transmitted to a motor vehicle that is at least partially automated. [7] Method according to one of the preceding claims, wherein the motion assumptions are determined on the infrastructure side, wherein at least one of the motion assumptions determined on the infrastructure side is wirelessly sent to a motor vehicle that is at least partially automated, wherein the Forward Reachable Set (401) is determined on the motor vehicle side based on the at least one motion assumption. [8] System (200) for determining a forward reachable set (401) of a road user, comprising: an infrastructure environment sensor system (203) which is set up to record an observation area of ​​a road traffic infrastructure (601) over an observation period, an investigative device (205) which is set up to determine movement assumptions of different types of road users located within the observation area over the observation period based on the recording, wherein the detection device (205) is set up to determine a Forward Reachable Set (401) of a road user currently located within the observation area of ​​the road traffic infrastructure (601) based on the movement assumptions determined of the road user type corresponding to the current road user. [9] Computer program (303) comprising instructions which, when the computer program (303) is executed by the system according to claim 8, cause it to execute a method according to any one of claims 1 to 7. [10] Machine-readable storage medium (301) on which the computer program (303) according to claim 9 is stored.

Images