Method for providing information about a traffic participant

Abstract

The invention relates to a method for providing information about traffic participants (T1, T2) within the environment of a vehicle, which is acquired by means of sensors of the vehicle itself. According to the invention, the acquired information is provided as a data structure (D) of vectors. Here, each vector corresponds to a cell (B1-Bn, BB1-BBm) of a predetermined, vehicle-fixed occupancy grid (B, BB), in which the traffic participant (T1, T2) described by the respective information is located. Each vector contains at least the coordinates of the corresponding belonging cell (B1-Bn, BB1-BBm), in which the respective traffic participant (T1, T2) is located, a velocity vector representing the velocity (v) of the respective traffic participant (T1, T2), a timestamp representing the time (t1, t2) at which the respective traffic participant (T1, T2) was acquired, and an object class representing the type of the respective traffic participant (T1, T2). The invention also relates to a method for operating an autonomously drivable vehicle (F1, F2).

Description

TECHNICAL FIELD

[0001] The invention relates to a method for providing information about traffic participants in the vehicle's surroundings, which is acquired by means of the vehicle's own sensors.

[0002] The invention also relates to a method for operating a vehicle which can be driven autonomously, highly autonomously or autonomously. BACKGROUND

[0003] From DE 10 2013 210 263 A1 a method for providing an occupancy map for a vehicle is known, in which the vehicle driving situation is ascertained from vehicle environment data acquired by means of a plurality of sensor devices by means of an ascertainment device, and the design of the occupancy map is adapted in accordance with the driving situation. The occupancy map has a plurality of cells arranged in a grid, which are adapted to the driving situation in accordance with the vehicle driving situation.

[0004] Furthermore, from DE 10 2010 011 629 A1 a vehicle environment presentation method is known, in which environment data is acquired and stored in a hierarchical data structure, and objects are identified in the environment. The object relevance for the application is ascertained and the degree of refinement of the hierarchical data structure is increased in regions in which objects with a high application-specific relevance are detected. In this case, the environment data is entered into an occupancy grid as sensor measurement data in order to obtain a probabilistic environment display. Each cell of the occupancy grid contains an occupancy probability which is calculated on the basis of the sensor measurement data here. SUMMARY

[0005] It is the task of the invention to specify a new method for providing information about traffic participants in the vehicle's surroundings, which is acquired by means of the vehicle's own sensors.

[0006] According to the invention, this task is accomplished by a method for providing information about traffic participants in the vehicle's surroundings, which is acquired by means of the vehicle's own sensors, having the features of claim 1 and a method for operating an autonomous vehicle having the features of claim 7.

[0007] Advantageous embodiments of the invention are the subject matter of the dependent claims.

[0008] In a method for providing information about traffic participants in the environment of a vehicle by means of information acquired by sensors of the vehicle, according to the invention, the acquired information is provided as a data structure representing a vector each, which vector depicts a respective traffic participant. Each vector here corresponds to a cell of a predetermined, vehicle-fixed occupancy grid, in which cell the traffic participant concerned is located. Each vector here comprises at least the coordinates of the corresponding cell of origin of the respective traffic participant in data form, a velocity vector representing the velocity of the respective traffic participant, a time stamp representing the acquisition time of the respective traffic participant and an object class representing the type of the respective traffic participant. The information about the traffic participants provided as vectors is preferably summarized in a data field and transmitted to a backend server.

[0009] The method allows the tracking of traffic participants over the entire so-called "operational design domain" (abbreviation: ODD). Here, when the information present on the backend server is provided to the vehicle fleet, a uniform picture of the traffic situation with a high degree of refinement in terms of temporal and spatial resolution can be generated for the entire vehicle fleet. The method here allows reliable route planning, collision avoidance as early as possible and, in turn, more stable driving of autonomous vehicles.

[0010] In one possible design of the method, the vehicle position and / or the vehicle orientation and / or occupancy grid information relating to the definition of the occupancy grid are transmitted to the backend server as additional information. This allows the backend server to reliably accurately convert the information obtained by the vehicle into a global coordinate system and thus provide it for use by other vehicles.

[0011] In another possible design of the method, the backend server performs a coordinate transformation for this purpose, by means of which the vectors received as data fields from the vehicles are transformed from the vehicle-fixed occupancy grid of the vehicle into a predetermined, position-fixed global occupancy grid.

[0012] In another possible design of the method, the vectors transformed into the global occupancy grid are provided to other vehicles for recall, so that they can use this information for their own driving.

[0013] In another possible design of the method, in the transmission made upon the call, the transformed vectors are transmitted as data fields from the backend server to at least one further vehicle, wherein each vector corresponds to a cell in which a traffic participant described by the respective information is located in the global occupancy grid. In this case, each vector at least comprises the coordinates of the corresponding belonging cell in which the respective traffic participant is located, a speed vector representing the speed of the respective traffic participant, a timestamp representing the acquisition time of the respective traffic participant and a target level representing the category of the respective traffic participant. This facilitates the tracking of traffic participants also in the entire "running design field" for other vehicles of a vehicle platoon in order to generate a uniform image of the traffic situation with a high degree of refinement in terms of temporal and spatial resolution. Reliable route planning, early collision avoidance and thus more consistent driving of autonomous vehicles can thus be achieved for other vehicles.

[0014] In another possible design of the method, the other vehicles are automatically operated in the form of a vehicle platoon designed for automated, in particular highly automated or autonomous operation.

[0015] In the method for operating an automated, in particular highly automated or autonomous, vehicle according to the application, information called from the backend server is taken into account in the automated operation of the vehicle. Automated, in particular highly automated or autonomous, driving of a vehicle necessarily requires knowledge of the vehicle's environment. For this purpose, a sufficient range of action of the vehicle's own sensors for acquiring the vehicle's environment and a correspondingly large field of view of the vehicle for reliable acquisition of the environment are required. In particular in urban environments, the range of action of the sensors and the field of view of the vehicle are therefore limited. With the aid of the method, this problem of a limited range of action of the sensors is solved in particular for autonomous vehicles of a vehicle platoon, i.e. vehicle environment information transmitted from at least one vehicle to the backend server is provided to the vehicles, so that they are supplied with information outside their own range of action of the sensors. As a result, the vehicles are not limited to the range of action of their sensors. Here, an environment model can be created for the vehicles within the scope of their motion trajectory planning based on the information called from the backend server, which goes beyond the range of action of their sensors and also includes regions that would be obscured for the sensors or lie outside their range of action in the motion trajectory planning. Long-term and optimized route planning can thus be carried out. In addition, the use of the backend server as an external source makes it possible to check the vehicle's own sensors, for example with regard to false sensor readings. With the aid of the information obtained from the backend server, the vehicles obtain additional information, which can be taken into account for checking the functionality of the vehicle's own sensors and identifying functionally limited sensors. BRIEF DESCRIPTION OF DRAWINGS

[0016] Embodiments of the application are explained in detail below with reference to the drawings, in which:

[0017] Figure 1 a perspective view of the vehicle and another road user at a first time instant and a perspective view of the occupancy grid at a further time instant together with the vehicle and the other road user,

[0018] Figure 2 an occupancy grid of an urban area is schematically shown,

[0019] Figure 3 a coordinate transformation of a vehicle-fixed occupancy grid and its content to a position-fixed global occupancy grid is schematically shown,

[0020] Figure 4 a position-fixed global occupancy grid together with two vehicle-fixed occupancy grids transformed into it of two vehicles is schematically shown,

[0021] Figure 5 providing information about road users in the vehicle's environment acquired by means of the vehicle's own sensors to a backend server and calling this information from the backend server is schematically shown. DETAILED DESCRIPTION

[0022] Parts that correspond to one another are provided with the same reference numerals in all figures.

[0023] Figure 1 a vehicle F1, a vehicle-centric occupancy grid B (also referred to as occupancy grid) of the vehicle F1 and a further road user T1 at a first time instant t1 are shown in a perspective view, and the occupancy grid B and the vehicle F1 and the further road user T1 at a second time instant t2 following the first time instant t1 are shown in a perspective view.

[0024] Here, the road user T1 moves with a speed v.

[0025] The vehicle F1 belongs, for example, to a vehicle platoon and is designed for automated operation, in particular highly automated operation or autonomous operation.

[0026] In such automated driving, the prediction of the behavior of the road user T1 is a major challenge. In complex traffic scenarios it is often difficult to track and predict the behavior of all sensed road users T1. High computational costs are also required for this.

[0027] An occupancy grid B is employed here for the prediction of the behavior, wherein the vehicle F1 has a vehicle-centric coordinate system with coordinates x, y, z. The x coordinate x always points forward here. The other axes depicted with the coordinates y, z are perpendicular thereto. A grid or a mesh is created around the vehicle F1 which is constructed statically around the vehicle F1 and moves with the vehicle F1 and lies in the acquisition area of the vehicle's own sensors, which are not shown in detail.

[0028] The vehicle Fl positions all traffic participants T1, e.g. pedestrians, cyclists, cars, trucks, buses, etc., with its sensors in its vehicle environment and determines their relative distance to the vehicle Fl. In addition to traffic participants, also stationary targets such as construction sites or other obstacles on the road are detected and positioned. The acquisition can here be performed by means of many sensors and / or a combination of different sensors, e.g. radar sensors, lidar sensors, camera sensors and / or ultrasound sensors, and corresponding data processing devices, e.g. deep learning algorithms.

[0029] The measured traffic participants T1 are depicted by data structures D representing vectors. The vectors representing the respective traffic participants T1 are here composed of information acquired by means of the vehicle's own sensors. Each vector here comprises as information the following data, which depict the position of the respective traffic participant T1 relative to the vehicle Fl in the form of the x, y and z coordinates x, y, z of the cells B1 - Bn of the occupancy grid B occupied by the traffic participant T1, wherein the z coordinate z describes the height of the respective traffic participant T1. In addition, the vector comprises a velocity vector representing the velocity v of the respective traffic participant T1, a timestamp representing the time t1, t2 at which the respective traffic participant T1 was acquired and a target type representing the type of the respective traffic participant T1. In addition, the vector can also contain, for example, the acquisition reliability based on the sensor from which the traffic participant T1 has been recognized, and other information, e.g. the intention of the traffic participant T1.

[0030] The occupancy grid B is a 2.5-dimensional grid, i.e. a two-dimensional grid, but here, for all traffic participants T1, the height is stored as z coordinate z in the respective vector. Here, the two-dimensional grid can be written as a matrix, wherein each cell B1 - Bn corresponds with its own x coordinate x and its own y coordinate y. To form the occupancy grid B, the two-dimensional grid is superimposed and / or combined with all the required information acquired by the vehicle's own sensors. That is, the data structures D depicting these traffic participants are combined into one data field and passed on to the backend server 1 as shown in Figure 5

[0031] The cells B1 - Bn of the occupancy grid B can here be occupied or unoccupied. At the time t1, the traffic participant T1 designed as a pedestrian is shown in the cell B1 of the occupancy grid B together with his data structure D depicting him. All other cells B2 - Bn are not occupied by a traffic participant T1, wherein the data structure D for the empty cells B2 - Bn is set to zero.

[0032] By combining the occupancy grid B with the data structures D of the respective traffic participants, an environmental display at the time t1 is generated.

[0033] ​According to the illustration, the traffic participant T1 moves from the cell B1 to the adjacent cell B2 between the two time instants t1, t2. Accordingly, at least the following vector changes, which depicts the position of the traffic participant T1 relative to the vehicle F1 in the form of the x, y and z coordinates a, y, z and is contained in the data structure D.

[0034] The entire occupancy grid B of the vehicle F1 can be calculated, for example, such that the matrix is multiplied by a global vector, which contains all vectors with the respective position of the traffic participant T1 relative to the vehicle F1 in the form of the x, y and z coordinates x, y, z. All unoccupied cells B1-Bn are set to zero.

[0035] The information about the behavior of the traffic participant T1 is collected during test drives and / or training drives of the autonomous vehicle F1. The information is stored here in a constant data stream in the form of a matrix. Here, inter alia, so-called deep learning models are employed to process the information to predict the future behavior of the traffic participant T1. For example, an artificial neural network with a deep learning model is created, which processes the information from the test drives and / or training drives. As more and more information is available, the model more accurately predicts the future behavior of the traffic participant T1 for a plurality of time instants t1, t2.

[0036] The aforementioned concept is also measured with respect to a metropolitan global occupancy grid gB having a uniform global coordinate system. Figure 2 Such a metropolitan global occupancy grid gB is illustrated.

[0037] Figure 3 The vehicle-fixed occupancy grid B and its content are coordinate-transformed to a position-fixed global occupancy grid gB (also referred to as operational computation domain, abbreviated as ODD). In the coordinate transformation, the entire vehicle-fixed occupancy grid B of the vehicle F1 is covered with a coherent grid, which is common to all autonomous vehicles F1, F2, in particular belonging to one vehicle platoon, in it. The vehicle F2 is in Figure 4 is not shown in detail.

[0038] The vehicle-centered coordinate system of the autonomous vehicle F1 containing the coordinates x, y, z and the corresponding occupancy grid B is transformed into a global coordinate system having the coordinates x', y', z'. The aim is to create a uniform global occupancy grid gB containing all information acquired by the sensors of all autonomous vehicles F1, F2 operating in the global occupancy grid gB. Each autonomous vehicle F1, F2 is associated with a global occupancy grid gB as Figure 5The shown back-end server 1 shares the information gathered by its sensors. The shared information contains data fields with data structures D depicting the vehicles F1, F2. Here, in the global occupancy grid gB only the occupied cells gB1 - gBm contain information. Each unoccupied cell gB1 - gBm obtains a zero value. Thus, the data stream can be minimized and is suitable for the extension of the operating design field containing a large vehicle fleet of autonomous vehicles F1, F2. A small latency can be achieved here, which is a main element for guaranteeing the operating capability.

[0039] The coordinate transformation ensures here that all information is preserved. Only the coordinate-dependent information in the respective data structure D, such as the coordinates of the cells and the velocity vectors, is transformed. The detected traffic participants T1 are shown in both the vehicle-centric occupancy grid B and the global occupancy grid gB.

[0040] As can be seen from the two transformations into the vehicle-fixed occupancy grid B, BB of the global occupancy grid gB and the two vehicles F1, F2 Figure 4 Each vehicle F1, F2 is assigned a separate vehicle-centric occupancy grid B, BB, respectively. Here, these vehicles F1, F2 can detect the same traffic participants T1, T2 or different traffic participants within the respective different vehicle-centric cells B1 - Bn, BB1 - Bbo.

[0041] By uploading their respective occupancy grid B, BB to the back-end server 1 and creating the global occupancy grid gB, the double-detected traffic participants T1, T2 occupy the same cells gB1 - gBm within the global occupancy grid gB based on the coordinate transformation. Here, the double-detected traffic participants T1, T2 obtain a higher reliability or higher gathering reliability, since they are detected independently of the two autonomously operating vehicles F1, F2.

[0042] Figure 5 The provision of the back-end server 1 with information gathered by means of the vehicles' own sensors about traffic participants T1, T2 in the vehicles' environment and the calling of said information from the back-end server 1 is shown.

[0043] After uploading all vehicle-centric occupancy grids B, BB to the back-end server 1 for the time t1, the global occupancy grid gB is calculated based on the coordinate transformation containing all traffic participants T1, T2.

[0044] The global occupancy grid gB is then downloaded onto all vehicles F1, F2 with respect to the artificial time stamp t1 *. Each vehicle F1, F2 obtains information about all moving targets in the global occupancy grid gB here. Stationary targets such as obstacles or worksites on the road can be considered in the same way.

[0045] With the additional information, each individual vehicle F1, F2 can check its own collected information and, for example, pre-plan the movement trajectory intersection of the traffic participants T1, T2 and extend its environmental model beyond its sensor range.

[0046] The global occupancy grid gB is here scaled only on the basis of a sufficient number of vehicles F1, F2 in the operating design domain. The global occupancy grid gB contains only information that has been collected by vehicles F1, F2. The cells gB1 - gBm under no vehicle-centric update are so marked that they are depicted as not detected cells gB1 - gBm for the specified time t1, t2.

[0047] The sensor information of the respective vehicle F1, F2 here contains all moving targets, in particular traffic participants T1, T2, which are merged in a global vector that contains all vectors including the respective position of the traffic participants T1, T2 relative to the respective vehicle F1, F2 in the form of x, y and z coordinates x, y, z.

[0048] In addition to the information about moving targets, additional data about the reliable operation of the vehicle platoon are also of interest, which are classified in a first category. In addition, non-critical information about infrastructure and traffic is also of interest, which are assigned to a second category.

[0049] The data contained in the safety-critical information of the first category are, for example, the traffic sign status sensed by the vehicle platoon, unusual static targets such as parked vehicles in the second row, accident vehicles, lost goods and the like. Non-critical information is also of extended interest, especially in modern cities. Information belonging to the second category are, for example, occupied parking spaces, potholes and all information about prospective infrastructure and city optimization.

[0050] In order to also provide the last-mentioned safety-critical and non-critical information, the vehicles F1, F2 of the vehicle platoon pool real-time information about the global information net. The information stored in the occupancy grids B, BB, gB can also be extended by requested data, for example, about empty parking spaces, pooled in the backend server 1 and distributed to suitable channels, for example, a parking App.

Claims

1. A method for providing information about traffic participants (T1, T2) within a vehicle environment, acquired by means of sensors on the vehicle itself. Its characteristics are, The collected information is provided as a data structure (D) representing corresponding vectors, each vector corresponding to a cell (B1-Bn, BB1-BBm) of a traffic participant (T1, T2) described by the corresponding information, located within a predetermined fixed vehicle occupancy grid (B, BB), and each vector contains at least the following data: - The coordinates of the corresponding traffic participant (T1,T2) in the cell (B1-Bn,BB1-BBm). - Represents the velocity vector of the corresponding traffic participant (T1, T2) (v). - Represents the timestamp of the time (t1,t2) when the corresponding traffic participant (T1,T2) was collected, and - Indicates the object category of the corresponding traffic participant (T1, T2); The data structure (D) is aggregated into a data field and transmitted to the backend server (1). As additional information, the following items are transmitted to the backend server (1): - The position of the vehicle (F1, F2), and / or - The orientation of the vehicle (F1, F2), and / or -Placement grid information relating to the definition of this placeholder grid (B,BB). The backend server (1) performs a coordinate transformation based on the additional information, in which the vector received as a data domain from the vehicle (F1,F2) is transformed from the fixed vehicle-attached placeholder grid (B,BB) on the vehicle (F1,F2) to a predetermined global placeholder grid (gB) with a fixed position.

2. The method according to claim 1, characterized in that, The vector transformed into the global placeholder grid (gB) is provided to other vehicles (F1, F2) for use.

3. The method according to claim 2, characterized in that, During the transmission at the time of the call, the transformed vector is transmitted as a data field from the backend server (1) to at least another vehicle (F1, F2), wherein each vector corresponds to the cell (gB1-gBo) of the global placeholder grid (gB) where the traffic participant (T1, T2) is located, as described by the corresponding information, and contains at least the following data: - The coordinates of the corresponding cell (gB1-gBo) where the traffic participant (T1,T2) is located. - Represents the velocity vector of the corresponding traffic participant (T1, T2) (v). - A timestamp, representing the time (t1, t2) when the corresponding traffic participant (T1, T2) was collected, and - Object category, which represents the type of the corresponding traffic participant (T1, T2).

4. The method according to claim 2 or 3, characterized in that, The other vehicles (F1, F2) operate automatically in a convoy designed for automatic, especially highly automatic or autonomous, operation.

5. A method for operating vehicles (F1, F2) capable of autonomous operation, wherein, According to claim 2, the information invoked by the backend server (1) is taken into account in the automated operation.

Images