METHOD FOR THE IDENTIFICATION OF ROAD USERS

Description

[0001] The invention relates to a method for detecting road users, in particular persons in road traffic. It is known from the prior art to detect road users using ADAS sensors such as cameras, radar, or lidar and to roughly classify them as persons, vehicles, or animals. Furthermore, it is known to track these road users, for example, for movement prediction.

[0002] For example, DE102014207802B3 describes a method and system for proactively detecting a road user's action. For this purpose, a camera captures a pixel-by-pixel structured image of the road user and generates corresponding image data.

[0003] US 2018 / 253595 A1 relates to a method for predicting the actions of road users. For this purpose, objects in the surrounding area are detected, their respective poses are recognized, and future behavior is derived from this.

[0004] WO 2019 / 116099 A1 relates to a method for predicting people's actions. For this purpose, the poses of detected people are analyzed and used to infer future movements of the respective person.

[0005] The disadvantage here, however, is that current recording and classification of people is not always sufficient, since people in road traffic can behave and move in a wide variety of ways.

[0006] It is therefore an object of the present invention to provide a method which enables a more accurate and robust detection and classification of persons.

[0007] This object is achieved by the subject matter of independent claims 1 and 4. Further advantageous embodiments are the subject matter of the subclaims.

[0008] In general, the interaction and communication options of road users are extremely complex. It is fundamentally possible to extend a classification approach so that not only rough classification results such as person or vehicle can be recognized, but also more specific classification results that describe, for example, more detailed characteristics of the person.

[0009] However, this approach is difficult to implement in practice because it is associated with a high effort in terms of the amount of data and the associated effort for data acquisition, labeling, etc., since the classification is usually applied to the raw data, e.g. camera image.

[0010] In particular, for safe trajectory planning, it is important to know whether a person is to be classified as a pedestrian or whether the person is moving, for example, using a means of transport.

[0011] According to the invention, a method for detecting road users, in particular persons in road traffic, is therefore proposed, comprising the following steps: Recording at least one environmental image or a sequence of environmental images using an optical sensor of a vehicle, detecting a person in the environmental image, determining a pose of the detected person; determining the presence of a means of transport of the person; assigning a movement profile to the person based on the determined means of transport of the person.

[0012] The optical sensor here is preferably a mono camera or a stereo camera system. The use of a surround-view camera system or a combination of the aforementioned alternatives is also conceivable. At least one of these cameras captures an image of the surroundings or a sequence of images. An algorithm that recognizes objects and people in images is then applied to this image of the surroundings or the sequence of images. Convolutional neural networks (CNNs) are suitable for this purpose. These CNNs can be trained, among other things, for person recognition in order to infer the identity of a person based on specific features in the image of the surroundings.

[0013] To determine a person's pose, a skeleton-like representation can be generated. This skeleton-like representation is generated by detecting key points and connecting the detected key points. The skeleton-like representation therefore represents the relative position and orientation of individual body parts of the pedestrian. This information can be used to determine a person's pose. For example, a comparison can be made between the detected pose and a look-up table to more quickly classify the person's orientation or posture.

[0014] To determine the presence of a means of transport, objects are considered that can be assigned to the person based on their proximity. Here, too, neural networks can be trained to identify certain objects as means of transport and distinguish them from, for example, carried luggage.

[0015] If a means of transport is unequivocally identified, a possible predefined movement profile can be assigned on this basis. Here, framework parameters can also be stored for specific means of transport, such as bicycles, skateboards, or inline skates, which are incorporated into the selection of the movement profile.

[0016] In the context of the present invention, the presence of a means of locomotion is determined by observing a time sequence of pose states or by observing individual images. Observing a time sequence of pose states is advantageous because, when using a specific means of locomotion, a pose is either permanently assumed or at least certain movement patterns and thus various poses are repeated. For example, a cyclist holds a certain pose. For a skateboarder, a certain movement pattern is repeated when a foot is used for propulsion, and thus certain poses are repeated multiple times within a certain period of time.

[0017] However, the presence of a means of locomotion can also be determined based on individual images. This is particularly possible with sensor data fusion, such as between a radar sensor and a camera sensor. The radar sensor, for example, can determine the speeds of certain limbs based on the available Doppler velocities. In combination with the camera image, this method can be used to identify a means of locomotion, since there is a specific leg movement pattern for the use of a means of locomotion, for example.

[0018] Furthermore, the means of transport is classified by a corresponding classifier. For this purpose, a classifier is trained to distinguish between, for example, an inline skater, a skateboarder, a regular pedestrian, or a cyclist, based on the pose and typical leg movements.

[0019] It is further preferred that the criticality of the person is determined based on the determined means of transport. This criticality describes, for example, the probability that the person will cross the roadway. Depending on the means of transport, changes of direction are possible more quickly. In conjunction with a higher speed than that of a normal pedestrian, a person using a corresponding means of transport represents a higher risk and must be assessed or classified accordingly in order to be able to adapt driving functions or intervention thresholds accordingly if necessary. In this case, for example, the driving speed of the vehicle can be reduced or an intervention threshold of an EBA system (emergency braking assistant) can be lowered if a higher criticality of the detected person is determined. Furthermore, the area with the detected person can be scanned more frequently orthe ROI (region of interest) can be enlarged to ensure the recognition of the person for a certain period of time or can be directed directly at the person.

[0020] In a particularly preferred embodiment, the predicted movement profile comprises a maximum speed, the person's degrees of freedom, and / or a direction of movement. These different parameters can be stored for each potentially recognizable means of transport, for example, in a look-up table. In the light of the invention, degrees of freedom refer to the various possibilities for changing direction with respect to the direction, the angle relative to the current direction of movement, and the speed of the change of direction. Depending on the means of transport, more or faster changes of direction are possible. For example, an inline skater can change direction with a larger angle more quickly than, for example, a cyclist.The movement profile also allows model-based tracking to be optimized, particularly with regard to the maximum achievable speeds associated with the vehicle. Furthermore, the vehicle can adapt more quickly to a change in the person's direction of movement, as a more accurate prediction of the person's position in the next time step is possible.

[0021] Furthermore, according to the invention, a system for detecting road users, in particular persons in road traffic, is proposed, comprising at least one optical sensor for recording an image of the surroundings or a sequence of images of the surroundings and a data processing device which is designed to to analyze the captured environmental image and perform person recognition, to perform pose recognition of the person, to determine the presence of a means of transport in the image and to assign a movement profile to the person based on the means of transport.

[0022] The data processing unit uses, for example, a CNN (Convolutional Neural Network). This enables the data processing unit to perform person and pose recognition with the help of trained classifiers. Furthermore, classifiers are trained that can be used to infer a specific means of transport based on certain characteristics.

[0023] A movement profile is then assigned to each of these means of transport.

[0024] Further advantageous embodiments and embodiments are apparent from the drawings, which show: Fig. 1a schematic flow diagram of an embodiment of the invention; Fig. 2 a schematic representation of an embodiment of the invention.

[0025] Figure 1 shows a schematic flow diagram of an embodiment of the invention. In step S1, an environmental image or a sequence of images is captured using at least one optical sensor 2. In step S2, a person is detected in this environmental image. In a further step S3, a pose of the detected person is determined. Furthermore, in step S4, the presence of a means of transport is determined. In a next step S5, a movement profile is assigned to the person based on the determined means of transport.

[0026] Figure 2shows a schematic representation of an embodiment of the invention. Here, a system 1 for detecting road users, in particular persons in road traffic, is shown in a schematic representation. The system 1 comprises an optical sensor 2 and a data processing device 3. The optical sensor 2 and the data processing device 3 are connected via a data connection D, by means of which image data can be transmitted from the optical sensor 2 to the data processing device. It would be conceivable for the data processing device to be connected to one or more actuators which control the vehicle accordingly based on the results of the data processing device. The data connection D is preferably designed as a wired connection. However, wireless connections such as WLAN, Bluetooth or similar would also be conceivable. List of reference symbols

[0027] 1System 2Optical sensor 3Data processing device DData connection S1-S5Process steps

Claims

1. Method for recognizing road users, in particular persons in road traffic, comprising the following steps: - recording (S1) at least one environment image or a sequence of environment images by means of an optical sensor (2) of a vehicle, - detecting (S2) a person in the environment image, - determining (S3) a pose of the detected person; - determining (S4) a presence of a means of transport of the person, wherein the presence of the means of transport is determined by considering a time sequence of pose states or by considering individual images, wherein the means of transport is classified by a corresponding classifier and the classifier classifies the means of transport on the basis of the pose and the typical leg movements; - assigning (S5) a movement profile to the person based on the determined means of transport of the person.

2. Method according to any of the preceding claims, characterized in that a criticality of the person is determined based on the determined means of transport.

3. Method according to Claim 1, characterized in that the predicted movement profile comprises a maximum speed, degrees of freedom of the person and / or a direction of movement.

4. System (1) for recognizing road users, in particular persons in road traffic, comprising at least one optical sensor (2) for recording an environment image or a sequence of environment images and also a data processing device (3) configured - to analyse the recorded environment image and to carry out person recognition, - to carry out pose recognition of the person, - to determine the presence of a means of transport in the image, wherein the presence of the means of transport is determined by considering a time sequence of pose states or by considering individual images, wherein the means of transport is classified by a corresponding classifier and the classifier classifies the means of transport on the basis of the pose and the typical leg movements; and - to assign a movement profile to the person based on the means of transport.