Method for detecting an overtaking maneuver by a neighboring vehicle in a traffic scene

Abstract

The invention relates to a method (100) for detecting an overtaking maneuver by a neighboring vehicle (2) in a traffic scene (3), comprising the following steps: - providing (101) image data of the traffic scene (3), wherein the image data is provided by at least one camera (5) of a vehicle (1), - providing (102) a region of interest (4) in the image data, wherein the region of interest (4) represents the neighboring vehicle (2), and temporary optical flow values ​​(6) for the region of interest (4) in the image data, - detecting (103) an extension of the region of interest (4) between at least two successive frames of the image data, - aggregating (104) the temporary optical flow values ​​(6) to approximate a movement of the neighboring vehicle (2), - detecting (105) the overtaking maneuver of the neighboring vehicle (2) based on the detection (103) and the approximate movement of the neighboring vehicle (2). Vehicle (2).Furthermore, the invention relates to a computer program, a device and a data storage medium for this purpose.

Description

The invention relates to a method for detecting an overtaking maneuver of a neighbouring vehicle in a traffic scene. The invention further relates to a computer program, a device and a data storage medium for this purpose.Prior ArtDriver assistance systems in vehicles may provide various functions by analyzing image data provided by cameras of the vehicle. For example, a distance to another vehicle traveling in front of the vehicle may be analyzed so that the vehicle may brake if the vehicle comes too close to or brakes in front of the vehicle.Furthermore, detection of an overtaking maneuver of another vehicle may play an important role for driver assistance systems in vehicles due to the unsafe situations that may be caused by the overtaking maneuver. Effective side monitoring with fast vehicle detection can thus be important. Missing the detection of an overtaking vehicle may result in serious consequences, such as a collision with the overtaking vehicle if the overtaking maneuver is performed. Otherwise, preventing the passing vehicle from returning to its own lane by performing excessive acceleration of the vehicle may also be dangerous.A classic structure-from-motion (SfM) approach, as known in the prior art, may have the following disadvantages in detecting passing vehicles. Availability of an optical flow of image corners may be reduced, 3D reconstruction for moving objects may be less reliable, and tracking may be difficult because a variable amount of the object may become visible in the traffic scene during passing.Disclosure of the InventionAccording to aspects of the invention there are provided a method having the features of claim 1, a computer program having the features of claim 8, a data processing apparatus having the features of claim 9 and a computer readable data storage medium having the features of claim 10. Further features and details of the invention are disclosed in the respective dependent claims, the description and the drawings. Features and details which are described in connection with the method according to the invention also correspond to the computer program according to the invention, the data processing device according to the invention and the computer-readable data storage medium according to the invention and in any case also vice versa.According to one aspect of the invention, there is provided a method of detecting an overtaking maneuver of a neighbouring vehicle, the method comprising the following steps:providing image data of a traffic scene, wherein the image data are provided by at least one camera of the vehicle,providing a region of interest in the image data, the region of interest representing the adjacent vehicle, and temporary optical flow values for the region of interest in the image data,detecting an extension of the region of interest between at least two successive frames of the image data,aggregating the temporary optical flow values to approximate a movement of the adjacent vehicle,detecting the passing maneuver of the adjacent vehicle based on the detecting, in particular detecting the extension of the region of interest, and the approximated movement of the adjacent vehicle.The method may be advantageous in that by detecting the passing maneuver, potential risks may be avoided. For example, if the passing maneuver is not detected, the vehicle may drive into the adjacent vehicle, either by changing to a respective lane of the adjacent vehicle or by accelerating while the adjacent vehicle is passing. The traffic scene can be a road with multiple lanes on which the passing maneuver takes place. The region of interest may represent the adjacent vehicle by a segmented set of pixels corresponding to the adjacent vehicle in the image data. Aggregating the temporary optical flow values may be performed by vector addition of the individual temporary optical flow values in the region of interest. A result of the aggregation of the temporary optical flow values may be a result vector of the aforementioned vector addition of the individual temporary optical flow values in the region of interest. The result vector may indicate the approximated motion of the points of the adjacent vehicle in the direction of travel of the vehicle, which in turn may indicate that the adjacent vehicle is moving away from the vehicle in the direction of travel. The passing maneuver may be identified if the region of interest extends between the at least two consecutive frames of the image data while aggregating the temporary optical flow values indicates that a result vector is pointing in the direction of travel.It is possible that the method further comprises the following steps:detecting the adjacent vehicle in the image data,segmenting the image data to provide the region of interest representing the adjacent vehicle.Detecting the adjacent vehicle and segmenting the image data to provide the region of interest may be performed by any desired image recognition algorithm known in the art, for example, using a convolutional neural network. The segmentation for providing the region of interest may be advantageous in that the temporary optical flow values may be provided based on the pixels of the region of interest, i.e. the movement of the pixels of the region of interest between the at least two consecutive single frames representing the adjacent vehicle.In another example, detecting the extension of the region of interest comprises the following step:comparing a set of pixels of the region of interest in a current frame with a set of pixels of the region of interest in a previous frame, wherein the current frame and the previous frame are a respective frame of the at least two consecutive frames of the image data.A higher amount of pixels of the region of interest in the current frame compared to the amount of pixels of the region of interest in the previous frame may be an indication of the passing maneuver of the adjacent vehicle.The temporary optical flow values may describe a movement of pixel values representing the region of interest between the at least two consecutive frames of the image data. In other words, the temporary optical flow values may describe a vector in the current frame that points from a coordinate in the image data of a respective pixel value in the previous frame towards a coordinate of the respective pixel value in the image data in the current frame.It can be advantageous if the detection and the aggregating take place based on exactly two successive individual images of the image data. This can be advantageous in that only two individual images are required to detect the passing maneuver.In one example, the detecting identifies the passing maneuver of the adjacent vehicle if the approximated motion of the adjacent vehicle indicates a forward motion of the adjacent vehicle relative to the vehicle. The forward movement may be a relative movement of the adjacent vehicle away from the vehicle in the direction of travel of the vehicle.The image data may be provided by at least two cameras of the vehicle. The method may then further comprise the following step:calculating depth information related to the adjacent vehicle based on a disparity between respective perspectives in the image data of the at least two cameras.The detecting of the passing maneuver may be further performed based on the calculated depth information related to the adjacent vehicle. Detecting the passing maneuver may thus include analyzing the calculated depth information to determine a distance and motion of the adjacent vehicle. The movement of the adjacent vehicle may be detected by comparing the depth information of the adjacent vehicle in the at least two frames of the image data.According to another aspect of the invention, there may be provided a computer program, in particular a computer program product comprising instructions which, when the computer program is executed by a computer, cause the computer to carry out the method according to the invention. Thus, the computer program according to the invention may have the same advantages as have been described in detail with reference to a method according to the invention.According to another aspect of the invention, a device for data transmission can be provided, which is designed to carry out the method according to the invention. As the means, for example, a computer executing the computer program according to the invention may be provided. The computer may include at least one processor that may be used to execute the computer program. In addition, a non-volatile data memory may be provided in which the computer program may be stored and from which the computer program may be read by the processor for execution.According to another aspect of the invention there may be provided a computer readable data storage medium comprising the computer program according to the invention and / or instructions which, when executed by a computer, cause the computer to carry out the steps of the method according to the invention. The data storage medium may be embodied as a data storage device, such as a hard disk and / or a non-volatile memory and / or a memory card and / or a solid state drive. The data storage medium can be integrated into the computer, for example.Furthermore, the method according to the invention can be implemented as a computer-implemented method.Other advantages, features and details of the invention will become apparent from the following description in which embodiments of the invention will be described in detail with reference to the drawings. In this connection, each of the features mentioned in the claims and in the description can be essential for the invention individually or in any combination. The following is shown: FIG. 1 : shows a method, a computer program, a data storage medium and a device according to embodiments of the invention, FIG. 2 : A vehicle with a camera and an adjacent vehicle according to embodiments of the invention, FIG. 3 : shows a single image (frame) of the image data according to embodiments of the invention, FIG. 4 : shows a subsequent image of the image data according to embodiments of the invention.FIG. 1 shows a computer program 20, a computer readable data storage medium 15 and a device 15 according to embodiments of the invention.FIG. 1 further shows a method 100 for detecting an overtaking maneuver of a neighboring vehicle 2 in a traffic scene 3 according to embodiments of the invention. In a first step 101, image data of the traffic scene 3 are provided, the image data being provided by at least one camera 5 of a vehicle 1. In a second step 102, there are provided: a region of interest in the image data, the region of interest 4 representing the adjacent vehicle 2, and temporary optical flow values 6 for the region of interest 4 in the image data. In a third step 103, an extension of the region of interest 4 between at least two successive frames of the image data is detected. In a fourth step 104, the temporary optical flow values 6 are aggregated in order to approximate a movement of the adjacent vehicle 2. In a fifth step 105, the passing maneuver of the adjacent vehicle 2 is detected, based on the detection 103 and the approximated movement of the adjacent vehicle 2.FIG. 2 schematically shows a vehicle 1 with a camera 5 and an adjacent vehicle 2 according to embodiments of the invention.FIG. 3 illustrates a frame (frame) of the image data according to embodiments of the invention. The single image shows a traffic scene 3, wherein a neighbouring vehicle 2 is visible. Further, temporary optical flow values 6 are shown inside a region of interest 4. The region of interest 4 may include the pixels representing the adjacent vehicle 2. The temporary optical flow values 6 may describe a movement of pixel values between successive individual images of the image data, as depicted in FIG. 3.FIG. 4 shows a subsequent image of the image data according to embodiments of the invention. The single image shows a traffic scene 3, wherein a neighbouring vehicle 2 is visible. Further, temporary optical flow values 6 have been aggregated into an aggregated temporary flow value 7, which is shown inside the region of interest 4.A method according to embodiments of the invention may be configured as follows. Based on semantic segmentation, a location of relevant vehicles, i.e., for example, of a neighboring vehicle 2, may be detected as a region of interest 4 (ROI) based on the image corresponding to a current frame. An aggregated measurement, at the pixel level, for temporary optical flow values 6 from this region of interest 4 can determine the object dynamics on a depth axis. If the region of interest 4 in a current frame is larger with respect to a previous frame, the dynamics may confirm that the object is traveling at a higher speed in the same direction as the vehicle 1, which may indicate that the adjacent vehicle 2 is performing an overtaking maneuver.A semantic segmentation approach may be constrained by the following factors. A semantic region of interest covering two vehicles may be erroneously characterized as only one vehicle. A semantic region of interest may not fully cover a vehicle during passing.There may be two areas of improvement that are particularly provided by the present invention. Neither optical flow nor semantic segmentation may provide complete information regarding detecting an overtaking maneuver, but fusion of both may advantageously increase the precision of the detection. What is particularly important in a passing context: In this method there may be no need for an accumulation of multiple individual images to determine whether the vehicle is passing, since advantageously only two individual images may be required.Using a set of two consecutive frames, in particular a current and a previous frame, semantic segmentation information for the vehicle 1 can be collected. In a first step, it can be detected whether the region of interest 4 of a neighboring vehicle 2 expands or expands. In particular, the size of the region of interest 4 in pixels in the current frame during an overtaking maneuver must be larger than in the previous frame. In a second step, temporary optical flow values 6 for the region of interest 4 of the adjacent vehicle 2 may be calculated or provided. In a third step, approximate dynamics corresponding to the passing maneuver may be determined by aggregating the temporary optical flow values to approximate dynamics of the neighboring vehicle 2, for example if it is moving forward and / or has a higher speed than vehicle 1.The flow aggregation formula may be, for example, the average of all vectors, i.e., of all temporary optical flow values 6, from the region of interest 4: wherein (du, dv) AGG is in particular the aggregated flow vector, i.e., the aggregated temporary optical flow value 7, (du, dv) i is in particular the measured temporary optical flow value 6, and N may be the number of measurements.The above explanation of the embodiments describes the present invention in the context of examples. Of course, individual features of the embodiments can be freely combined with each other provided that it is technically reasonable without departing from the scope of the present invention.

Claims

Method (100) for detecting an overtaking maneuver of a neighbouring vehicle (2) in a traffic scene (3), comprising the following steps: - providing (101) image data of the traffic scene (3), wherein the image data are provided by at least one camera (5) of a vehicle (1), - providing (102) a region of interest (4) in the image data, wherein the region of interest (4) represents the neighbouring vehicle (2), and temporary optical flow values (6) for the region of interest (4) in the image data, - detecting (103) an extension of the region of interest (4) between at least two successive frames of the image data, - aggregating (104) the temporary optical flow values (6) to approximate a movement of the neighbouring vehicle (2), detecting (105) the passing maneuver of the adjacent vehicle (2) based on the detecting (103) and the approximated movement of the adjacent vehicle (2).The method (100) according to claim 1, characterized in that the method (100) further comprises the steps of: - detecting the adjacent vehicle (2) in the image data, - segmenting the image data to provide the region of interest (4) representing the adjacent vehicle (2).The method (100) according to any of the preceding claims, characterized in that detecting (103) the extension of the region of interest (4) comprises the step of: - comparing a set of pixels of the region of interest (4) in a current frame with a set of pixels of the region of interest (4) in a previous frame, wherein the current frame and the previous frame are a respective frame of the at least two consecutive frames of the image data.Method (100) according to any of the preceding claims, characterized in that the temporary optical flow values (6) describe a movement of pixel values representing the region of interest (4) between the at least two successive frames of the image data.Method (100) according to one of the preceding claims, characterized in that the detecting (103) and the aggregating (104) are carried out based on exactly two successive individual images of the image data.Method (100) according to one of the preceding claims, characterized in that the detection (103) identifies the passing maneuver of the adjacent vehicle (2) if the approximated movement of the adjacent vehicle (2) indicates a forward movement of the adjacent vehicle (2) relative to the vehicle (1).Method (100) according to any of the preceding claims, characterized in that the image data are provided by at least two cameras (5) of the vehicle (1) and the method (100) further comprises the step of: - calculating depth information relating to the adjacent vehicle (2) based on a disparity between respective perspectives in the image data of the at least two cameras (5), wherein the detecting (103) of the passing maneuver is further performed based on the calculated depth information relating to the adjacent vehicle (2).A computer program product (20) comprising instructions which, when the computer program (20) is executed by a computer (10), cause the computer (10) to carry out the method (100) of any preceding claim.Data processing device (10) comprising means for carrying out the method (100) according to one of claims 1 to 7.A computer readable storage medium (15) comprising instructions which, when executed by a computer (10), cause the computer (10) to carry out the steps of the method (100) of any one of claims 1 to 7.

Images