Method and device for determining the motion state of an object, and device for controlling a vehicle system

Abstract

The invention relates to a method for determining the motion state of at least one object in a vehicle environment. The vehicle has at least one vehicle camera for providing image data representing the environment. The method comprises the step of reading in a motion measure and quality information generated by processing the image data. The method also comprises the step of generating pixel-specific quality information using the read-in quality information. The method further comprises the step of determining a motion probability for each pixel using the motion measure and the pixel-specific quality information. The motion probability indicates for each pixel the probability that the pixel belongs to a moving object or to a static object as a motion state. The invention further relates to a method for controlling a vehicle system of a vehicle. The invention further relates to a device, a computer program and a machine-readable storage medium.

Description

Technical Field

[0001] The present invention is based on a method and a device for determining a motion state of an object in the environment of a vehicle and a device for controlling a vehicle system of a vehicle. The subject matter of the present invention also includes a computer program. Background Art

[0002] For example, driver assistance functions and functions for autonomous driving require, in particular, the most precise possible sensing and description of the vehicle environment. For example, the recognition of self-moving objects, such as people, cyclists, or other vehicles, can be a component of this. Cameras are particularly suitable as sensor types for environmental sensing, as they enable the acquisition of high-resolution motion. This can be achieved, for example, by analyzing image sequences using correspondence methods such as so-called optical flow. For example, geometric methods can be used to identify self-moving objects. Self-moving objects can be identified based on the fact that the motion of the object observed in the image sequence (optical flow) cannot be explained by the motion of the camera. However, the results of the above-mentioned correspondence methods can be noisy or systematically erroneous, which can lead to incorrect or missing dynamic object detection results. Summary of the Invention

[0003] Against this background, the solution presented here provides a method and a device for determining the motion state of an object in a vehicle environment, which device uses this method, and finally a corresponding computer program. Advantageous expansions and improvements of the method and device described in the present invention can be achieved through preferred embodiments.

[0004] According to specific embodiments, camera-based detection of moving objects in the vehicle environment is particularly possible, wherein a dynamic object probability or motion probability (DOP) and an associated confidence level can be determined for each identified correspondence of pixels in successive images. To reduce noise effects, two algorithmic approaches can be provided, for example, wherein one algorithmic approach primarily eliminates random noise effects and the other algorithmic approach primarily eliminates systematic noise effects.

[0005] Because random noise effects can often originate from correspondence algorithms, such as optical flow, compensation for these effects can be achieved by additionally knowing the quality or quality of the correspondence. If the quality of a correspondence measurement is poor, then such a measurement should not be classified, or only with a small probability, as belonging to a dynamic object, as it could be an erroneous or noisy measurement. Known correspondence algorithms can also provide quality information for each measurement, indicating how good, accurate, or likely the corresponding measurement is. This can be achieved, for example, in the form of a confidence level, standard deviation, or abstract metric. For example, optical flow can provide quality information that can describe measurement quality, ambiguity probability, and temporal consistency. Depending on the embodiment, compensation for systematic noise effects can be achieved, for example, using a separate algorithmic approach. In particular, the temporal consistency between dynamic objects and the static real world can be exploited. This is based on the assumption, for example, that a correspondence that has been classified as static for a long time is less likely to suddenly become dynamic, or that pixels previously classified as dynamic are more likely to be dynamic in the future than static. To exploit temporal consistency, a cascade of correspondences can be determined over a time period that is longer than the time period between two consecutive images.

[0006] Advantageously, embodiments can provide a robust system for detecting self-moving or dynamic objects based on a camera. This system enables reliable camera-based detection of dynamic objects and, in doing so, improves robustness against the aforementioned noise effects. Consequently, interference effects caused by random and systematic noise during use in real products can be compensated for using a unified approach. This can be achieved by initially converting, for example, geometrically based measures of motion or dynamic objects, such as epipolar errors, into motion probabilities, which can have a particularly advantageous effect because, compared to geometrically based measures, the temporal integration of motion probabilities is mathematically and physically well-optimized. Unlike cumulative measures such as epipolar errors, these motion probabilities do not vary easily and cannot have zero crossings, depending on the position in the image, the camera's own motion, object motion, etc. These motion probabilities can describe abstract probability variables that can, for example, lie in a range of values ​​between zero and one and can therefore always be interpreted identically.

[0007] A method for determining a motion state of at least one object in the environment of a vehicle is proposed, wherein the vehicle has at least one vehicle camera for providing image data representing the environment, wherein the method comprises the following steps:

[0008] - reading in a motion measure and quality information generated by processing the image data, wherein the motion measure comprises continuous measurement values ​​generated by using a correspondence identified by a correspondence algorithm between pixels in successive images represented by the image data for detecting moving object pixels, and wherein the quality information indicates the quality of the correspondence at least with respect to interference influences caused by noise;

[0009] - generating pixel-specific quality information using the read-in quality information, wherein the pixel-specific quality information indicates the quality of the assignment for each pixel; and

[0010] Using the motion measure and the pixel-specific quality information, a motion probability is determined for each pixel, wherein the motion probability indicates for each pixel the probability that the pixel belongs to a moving object or to a stationary object as a motion state.

[0011] The method can be implemented, for example, in software or hardware, or in a hybrid of software and hardware, for example, in a controller or device. The vehicle can be a motor vehicle, such as a passenger car or a commercial vehicle. In particular, the vehicle can be a highly automated vehicle or a vehicle for highly automated driving. The at least one vehicle camera can be mounted or fixedly installed in the vehicle. The motion state can be represented by at least one numerical value that indicates a static motion state, a dynamic motion state, and, in addition or alternatively, a motion state between static and dynamic with an ascertained probability. The correspondence algorithm can be, for example, so-called optical flow or the like. The correspondence can be identified between temporally consecutive images in the image data. The correspondence can be, for example, point correspondence or correspondence between lines, surfaces, or more complex shapes in the image. Based on the correspondence, a suitable motion measure or dynamic object measurement can be derived to indicate the dynamic motion state. In this case, it can be inferred from the image correspondence whether the correspondence belongs to the static real world or to a moving object. Such a measure can be, for example, an angular deviation from an epipolar line. The motion measure can be generated using the correspondence and other processing rules. To this end, motions that do not correspond to the expected behavior of the static real world can be detected, such as motions that violate the epipolar pairs, motions that are outside the expected range of point correspondences in the static real world, and / or motions that are far from the camera and that would result in negative depths in triangulation. Motions that violate additional assumptions, such as the planarity assumption, can also be detected. The motion measure can represent a measure for detecting pixels of dynamic objects or a measure for identifying dynamic objects. The quality information can represent the (co)variance of the correspondence or the correspondence algorithm, a distribution density function, or an abstract measure of quality, such as reliability, robustness, or similar.

[0012] Furthermore, the method can include a step of performing a temporal filtering of the motion probabilities using the correspondence in order to generate filtered motion probabilities that represent the motion state.

[0013] In this case, in an implementation step, a confidence value can be determined for each filtered motion probability. This confidence value can indicate how reliable the filtered motion probability is and, additionally or alternatively, how accurately it describes the motion state.

[0014] According to one embodiment, during the implementation step, the filtered motion probability can be binarized using a threshold comparison. The binarized filtered motion probability can indicate the motion state as static or dynamic. This embodiment offers the advantage of more reliably determining whether a pixel belongs to a static object or a moving or dynamic object, and thus whether a static or dynamic object is present in the vehicle environment. Furthermore, the number of objects incorrectly identified as "dynamic" can be further reduced.

[0015] In the implementation step, temporal filtering can also be performed using a recursive filter and, in addition or alternatively, a T memory element. The recursive filter can be a first-order recursive filter. The T memory element can be a T flip-flop. This embodiment has the advantage that temporal filtering can be performed simply and reliably on the previously determined motion probability.

[0016] Furthermore, in an implementation step, the filtered motion probabilities generated in at least one previous time step can be merged with the filtered motion probabilities generated in the current time step and accumulated in addition or alternatively. The filtered motion probabilities generated in at least one previous time step can be stored. The filtered motion probabilities generated in at least one previous time step can also be transformed into the current time step using the correspondence. This transformation can also be referred to as a so-called warp. This embodiment offers the advantage that a low-pass behavior can be achieved, which ensures that short-term measurement anomalies are not mistakenly interpreted as dynamic objects.

[0017] In this case, fusion can be performed using weighted values, and additionally or alternatively, accumulation can be performed to weight the filtered motion probabilities generated in at least one previous time step. The weighted values ​​can be adjusted based on how many previous time steps the filtered motion probabilities generated in at least one previous time step have been accumulated over. This embodiment has the advantage that the weighted values ​​obtained in this way can be used as a quality criterion for the fused motion probabilities.

[0018] Furthermore, an error propagation method can be used in the generation step and, in addition or as an alternative, a machine-learned model. The error propagation method can be a Gaussian error propagation method. This embodiment has the advantage that random noise effects can be reliably and accurately compensated to avoid false detection results. It can advantageously be checked whether a measurement result showing dynamic behavior is more likely to be caused by noise or by actual motion.

[0019] A method for controlling a vehicle system of a vehicle is also proposed, wherein the method comprises the following steps:

[0020] - determining the motion state of at least one object in the vehicle environment according to one embodiment of the above method;

[0021] Using the movement state, a control signal is generated for output to a vehicle system in order to actuate the vehicle system.

[0022] The vehicle system may be a driver assistance system, a control system for highly automated driving, and additionally or alternatively another vehicle system for controlling vehicle functions as a function of detected objects in the vehicle surroundings.

[0023] The solution proposed here also proposes a device, which is configured to implement, control or realize the steps of the variant of the method proposed here in a corresponding device. The object on which the invention is based can also be quickly and effectively achieved by the embodiment variant of the invention in the form of a device.

[0024] To this end, the device can include: at least one computing unit for processing signals or data; at least one memory unit for storing signals or data; at least one interface to a sensor for reading in sensor signals from the sensor or at least one interface to an actuator for outputting data signals or control signals to the actuator; and / or at least one communication interface for reading in or outputting data embedded in a communication protocol. The computing unit can be, for example, a signal processor, a microcontroller, or the like, wherein the memory unit can be a flash memory, an EEPROM, or a magnetic memory unit. The communication interface can be designed for wireless and / or wired reading in or outputting data, wherein a communication interface capable of reading in or outputting wired data can, for example, electrically or optically read in this data from or output this data to a corresponding data transmission line.

[0025] Currently, a device can be understood as an electrical appliance that processes sensor signals and outputs control signals and / or data signals accordingly. The device can have an interface, which can be constructed in hardware and / or software. In the case of a hardware construction, the interface can be, for example, part of a so-called system ASIC that contains the various functions of the device. However, it is also possible that the interface is an integrated circuit of its own or is at least partially composed of discrete components. In the case of a software construction, the interface can be, for example, a software module that exists on a microprocessor alongside other software modules.

[0026] Advantageously, a computer program product or a computer program with a program code is also proposed, which can be stored on a machine-readable carrier or storage medium, such as a semiconductor memory, a hard disk memory or an optical memory, and in particular when the program product or program is executed on a computer or a device, the program code is used to implement, realize and / or control the steps of the method according to one of the aforementioned embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] An exemplary embodiment of the solution proposed here is shown in the drawings and explained in detail in the following description.

[0028] Figure 1 a schematic diagram of a vehicle having an apparatus according to one embodiment;

[0029] Figure 2 A flowchart of a method for determining according to one embodiment;

[0030] Figure 3 A flow chart of a method for controlling according to one embodiment;

[0031] Figure 4 A schematic diagram of an apparatus according to an embodiment;

[0032] Figure 5 exemplary imaging of processed image data;

[0033] Figure 6 Exemplary imaging of image data processed by means of a device according to an embodiment;

[0034] Figure 7 Exemplary imaging of image data processed by means of an apparatus according to an embodiment; and

[0035] Figure 8 Exemplary imaging of image data processed by means of a device according to one embodiment. DETAILED DESCRIPTION

[0036] Before discussing the embodiments in more detail below, the background of these embodiments is briefly explained.

[0037] The results of correspondence methods, such as optical flow, can often be noisy. This noise can lead to correspondences at actually static scene points being misinterpreted as dynamic. Such misinterpretations are particularly undesirable in the context of driver assistance or autonomous driving, as they can lead to unwanted and unpredictable system reactions, such as inappropriate triggering of emergency braking. This correspondence noise can be further divided into a random component, such as measurement noise in the correspondence algorithm, image data noise, and a systematic component. The latter presents a major challenge, as it can often lead to large deviations or outliers in the correspondences and can therefore be interpreted as dynamic objects. This systematic noise effect is often caused by image texture, such as periodic structures in the image, weakly textured surfaces in the image, or aperture problems, which can often occur along lines and sharp texture edges in the image. However, effects that already occurred during image acquisition and violate certain standard assumptions of the subsequent algorithm chain are also relevant. These assumptions include, for example, 1) simultaneous exposure of all pixels in the image for the same duration (global shutter with constant or linear exposure) and 2) a known geometric transformation from a 3D point in the real world to its projection onto the 2D image surface (intrinsic calibration). Assumption 1) is violated, for example, when capturing images with high dynamic range (HDR), where various parts of the image are exposed for different durations, depending on their brightness. These differences in exposure time can result in the associated correspondences being of different lengths. Similarly, rolling shutter cameras can violate assumption 1) because the image is exposed line by line and thus distorted when the camera and / or the object moves. Assumption 2) is violated, for example, if optical elements, such as windshields, leave additional distortion in the image that is unknown to the camera manufacturer during the calibration process. Overall, it can be seen that conventional detection methods for dynamic objects can be susceptible to various interference or noise effects, which can be minimized or avoided according to embodiments. In particular, when using commercially available and particularly cost-effectively mass-produced cameras, the last-mentioned systematic effects are challenges that can be addressed according to exemplary embodiments.

[0038] In the following description of advantageous exemplary embodiments of the present invention, identical or similar reference numerals are used for similarly acting elements that are shown in the various figures, wherein a repeated description of these elements is omitted.

[0039] Figure 1According to one exemplary embodiment, a schematic diagram of a vehicle 100 with a device 120 is shown. Vehicle 100 is a motor vehicle, such as a land vehicle, in particular a passenger car, truck, or other commercial vehicle. In the environment of vehicle 100, only one object X is arranged, by way of example. According to the exemplary embodiment shown here, vehicle 100 includes, by way of example, only one vehicle camera 102, a processing device 110, a device 120, a generating device 130, a controller 140, and, by way of example, only one vehicle system 150.

[0040] Vehicle camera 102 is configured to record or sense the environment of vehicle 100, and thus also object X. Furthermore, vehicle camera 102 is configured to provide image data 105, which represent the environment of vehicle 100, and thus also object X. Processing device 110 is configured to, using image data 105, identify correspondences between pixels in successive images represented by image data 105 by means of a correspondence algorithm. Furthermore, processing device 110 is configured to, using this correspondence, generate a dynamic object measure 117 as a continuous measurement value for detecting moving object pixels and to generate quality information 119 indicating the quality of the correspondence. Processing device 110 is also configured to provide generated dynamic object measure 117 and quality information 119.

[0041] Device 120 is configured to determine a motion state of at least one object X in the environment of vehicle 100. Device 120 is also configured to read in generated motion measure 117 and quality information 119 from processing device 110. Device 120, or a determination device, is configured to provide a state signal 125 using motion measure 117 and quality information 119, which indicates or represents a determined motion state of at least one object X in the environment of vehicle 100. Device 120 will be discussed in greater detail with reference to the following figures.

[0042] According to the exemplary embodiment shown here, device 120 and generating device 130 are part of a controller 140. Controller 140 is configured to control at least one vehicle system 150 of vehicle 100. To this end, generating device 130 is configured to generate a control signal 135 using the motion state represented by state signal 125 for output to vehicle system 150 in order to control vehicle system 150. Vehicle system 150 may be a driver assistance system, a control system for highly automated driving, or another vehicle system.

[0043] Figure 2According to one embodiment, a flow chart of a method 200 for determining is shown. The method 200 for determining can be implemented to determine the motion state of at least one object in the vehicle environment. Figure 1 Thus, the method 200 for determining can be implemented in conjunction with a vehicle having at least one vehicle camera for providing image data representative of the environment. The method 200 for determining can also be implemented in conjunction with a vehicle having at least one vehicle camera for providing image data representative of the environment. Figure 1 Vehicle 200 for determining comprises an input step 210 , a generation step 220 , an ascertainment step 230 and optionally an execution step 240 .

[0044] In a read-in step 210, a motion measure and quality information generated by processing the image data are read in. The motion measure comprises a continuous measurement value generated by using correspondences between pixels in successive images represented by the image data, identified by a correspondence algorithm, to detect moving object pixels. The quality information indicates the quality of the correspondence, at least with respect to interference effects caused by noise.

[0045] In a generation step 220, pixel-specific quality information is generated using the read-in quality information. The pixel-specific quality information indicates the quality of the correspondence for each pixel. In a determination step 230, a motion probability is determined for each pixel using the motion measure and the pixel-specific quality information. The motion probability indicates, for each pixel, the probability of belonging to a moving object or a stationary object as a motion state.

[0046] In an implementation step 240 , the motion probabilities are temporally filtered using the correspondences in order to generate filtered motion probabilities that represent a motion state.

[0047] Figure 3 According to one embodiment, a flow chart of a method 300 for controlling is shown. The method 300 for controlling can be performed to control a vehicle system of a vehicle. Figure 1 In addition, the method 300 for manipulating can be implemented in conjunction with a vehicle or a similar vehicle. Figure 1 The method 300 for controlling comprises a determination step 310 and a generation step 320 .

[0048] In a determination step 310, a motion state of at least one object in the vehicle environment is determined. The determination step 310 has Figure 2In a generating step 320 , a control signal is generated to be output to a vehicle system using the motion state determined in the determining step 310 in order to control the vehicle system.

[0049] Figure 4 A schematic diagram of a device 120 is shown according to one embodiment. Here, the device 120 is equivalent to or similar to Figure 1 In addition, Figure 4 Also shown in Figure 1 The processing device 110 is equivalent to or similar to the processing device Figure 1 The generating device 130 is equivalent to or similar to the generating device of Figure 1 The vehicle system is comparable or similar to the vehicle system 150.

[0050] Processing device 110 includes an image acquisition unit 412, a correspondence algorithm unit 414, a visual odometry unit 416, and a motion measurement unit 418. Processing device 110 is configured to apply a geometric detection method to situations where the camera is moving. The correspondence between temporally consecutive images is used as the basis for the geometric detection method for detecting moving objects.

[0051] Image capture unit 412 is configured to capture images based on image data from a vehicle camera, or in other words, to capture temporally consecutive images. Correspondence algorithm unit 414 is configured to apply a correspondence algorithm, such as optical flow, to the captured images. In other words, correspondence algorithm unit 414 is configured to determine or identify point correspondences between images, for example, by using optical flow and / or tracking significant image points or pixels. Instead of point correspondences, correspondences between lines, surfaces, or more complex shapes in the images can also be determined. Correspondence algorithm unit 414 is also configured to provide quality information 119 indicating the quality of the correspondence or the quality of the ascertained correspondence, as well as to provide correspondence data 415. Visual odometry unit 416 is configured to determine the vehicle camera's own motion based on the correspondence or point correspondences. Visual odometry unit 416 is also configured to provide additional quality information 419 indicating the quality of the own motion or the quality of the ascertained own motion.

[0052] Motility measure unit 418 is designed to generate and provide a motility measure 117. Motility measure 117 represents a measure generated for each pixel in order to detect dynamic objects, such as an angle of epipolar violation. To this end, motions that do not correspond to the expected behavior of the static real world are detected with the aid of motility measure unit 418, such as motions that violate epipolarity, such as objects crossing, epipolar-compliant motions that lie outside the expected range of point correspondences in the static real world, so-called flow vector bounds, such as fast oncoming objects, and / or epipolar-compliant motions that are moving away from the camera and would result in a negative depth, so-called negative depth constraint, under triangulation, such as objects overtaking. Motility measure unit 418 is also used to detect motions that violate additional assumptions, such as the planarity assumption, meaning that objects actually move on a flat surface; however, the triangulation of dynamic objects pushes these dynamic objects below or above the relevant plane.

[0053] Here, the mobility measure unit 418 generates at least one of the listed measures for detecting dynamic object pixels (episode violation, boundary flow vector, negative depth constraint, planarity-related measures) using the mobility measure 117 or so-called dynamic object measurement. However, other measures for identifying dynamic objects can also be calculated in addition. Similarly, measures that are not obtained geometrically but based on data, that is, by machine learning, are conceivable. If data from other sensor types, such as laser scanners, radar, digital maps, etc., can also be projected into the current camera image, these data can also be used here. It should be noted that the mobility measure 117 is generated and provided not only in the form of a binary decision, that is, whether the pixel is static or dynamic, but also in the form of multi-level measurement variables, such as the angle of the epipolar violation, the length of the flow vector, the value of the negative depth, the distance to a reference plane, or similar variables.

[0054] Device 120 is designed to determine the motion state of at least one object in the vehicle environment. Device 120 has a generating device 422, a determining device 424, and an executing device 426. In addition, according to the exemplary embodiment shown here, a plurality of T memory elements T to T are shown. n Furthermore, a binary signal generating unit 428 is shown, which the device 120 may optionally also have. The device 120 is designed to read in the motion measure 117 and the quality information 119 from the processing device 110. The device 120 is also designed to read in the additional quality information 419 and the correspondence data 415.

[0055] The generating device 422 is designed to generate pixel-specific quality information 423 using the read-in quality information 119 and optionally additional quality information 419. The pixel-specific quality information 423 indicates the quality of the correspondence for each pixel. In other words, the generating device 422 is designed to convert the quality information 119 and 419 into the quality of the dynamic object measure or the quality of the motion measure 117, for example, using a lookup table. The determining device 424 is designed to determine the motion probability P for each pixel using the read-in motion measure 117 and the pixel-specific quality information 423. k . Movement probability P k For each pixel, the probability that the pixel belongs to a moving object or to a stationary object is displayed. In other words, the determination device 424 is designed to take into account random noise in the optical flow, wherein the motion probability P is derived from the motion measure 117. k The implementation device 426 is designed to use the correspondence or the correspondence data 415 to determine the motion probability P k Temporal filtering is performed to produce filtered or fused motion probabilities The filtered or fused motion probabilities indicate the motion state. In other words, the implementation device 426 is designed to take systematic noise into account, wherein the robustness against windshield distortion, HDR effects, etc. is increased by temporal filtering.

[0056] Compensation for random noise effects can be achieved with the aid of a generating device 422 and an determining device 424. As a countermeasure to random noise effects, quality information 119 of a correspondence algorithm, such as optical flow, or quality information of a correspondence algorithm unit 414 is taken into account, and optionally additional or further quality information 419 of a visual odometry method or visual odometry unit 416 is also taken into account. This information can exist, for example, in the form of (co)variance, a distribution density function, or an abstract measure. The generating device 422 is configured to convert the quality information 119 and / or 419 so that this quality information describes the quality of the measured motion measure 117 or the dynamic object measurement for each pixel. Error propagation methods, such as the Gaussian error propagation method, or learned model relationships or machine learning can be used. As a result, specific or pixel-specific quality information 423 is also known for each motion measure 117, for example, quality information in the form of a variance or a distribution density function. Next, random noise effects are compensated by means of an ascertainment device 424 which is designed to derive a motion probability P for each pixel as a function of the motion measure 117 and the corresponding quality information 423 in the following manner: k, whether the pixel represents a dynamic object. This checks whether the measurement result is more likely due to noise or actual motion. For example, if a 1-degree epipolar violation angle is measured at a pixel, and the corresponding optical flow there results in a standard deviation of 0.1 degrees, then the probability that the pixel represents a dynamic object is high. However, if the standard deviation is higher, such as 0.8 degrees, then the probability that the pixel represents a dynamic object is significantly lower.

[0057] By means of the implementation device 426 and optionally by means of the binary signal generation unit 428, systematic noise effects can be compensated, which will be discussed further below. The compensation of systematic noise effects exploits the temporal consistency of moving objects and the static real world and assumes that the transition between two images occurs slowly, that is, not abruptly from image to image, but more slowly over a series of images. By means of the previously determined motion probability P k Temporal filtering of , leaves behind a low-pass feature that is responsible for not misinterpreting short-term measurement outliers as dynamic objects.

[0058] According to one embodiment, the temporal filtering is performed in such a way that at least the filtering results of the previous time step are stored, for which purpose the device 120 has a plurality of T memory elements T to T n , and this T memory element at the current time point is merged with the current motion probability in the implementation device 426. For this purpose, the result from the previous time step is first transformed into the current time step by means of the correspondence or correspondence data 415, this transformation is also called warping.

[0059] According to one embodiment, the motion probability P is implemented by a first-class recursive filter k Temporal filtering, fusion or accumulation. Here, the current motion probability P K and the previously warped fusion result Accumulate for each pixel in, is a weighted or weighted value that, for example, accounts for outdated measurements Count how many times it has accumulated in time. As with the previous motion probability, here too, the previous weighted value is warped into the current time step: After fusion, the current weight value is incremented: If there is no current measurement for a pixel, for example because no flow vector can be determined there, the weighting value is reset to w k = 0. According to one embodiment, the synthesized weighted value w k Used as filtered or fused motion probability A high weighted value means that the corresponding motion probability has been confirmed many times. Therefore, the confidence of this measurement is relatively high.

[0060] According to the exemplary embodiment shown here, the device 120 also has a binary signal generating unit 428. The binary signal generating unit 428 is designed to carry out a filtered motion probability calculation using a threshold comparison. The binary filtered motion probability shows the motion state as static or dynamic. Therefore, depending on the application, a binary judgment can be made for each pixel as static / dynamic. This binary judgment can be made, for example, by simply applying the extreme value to the motion probability However, more complex operations are also conceivable, such as spatial aggregation in the image, motion segmentation, object detection, etc. It should be noted here that the combination of the motion probability and the associated confidence measure is an improved measure for detecting dynamic objects compared to conventional approaches.

[0061] Device 120 is designed to output state signal 125 to generating device 130. Generating device 130 is also designed to carry out detection of dynamic objects. Finally, vehicle system 150 can be controlled in this way.

[0062] Figure 5 An exemplary image 500 of processed image data is shown. Here, the image data is obtained by Figure 1 or Figure 4 Furthermore, in the exemplary imaging 500 , a first object X1 , a second object X2 , and a third object X3 are shown. Figure 5 The example shows which pixels are classified as dynamic when only one measure of mobility, such as the epipolar violation angle, is compared with an extreme value. It is clear that in addition to the correctly identified "dynamic" objects, such as the crossing cyclist, the second object X2, and the oncoming vehicle, and the third object X3, there are also many false positive detections, or pixels that are incorrectly classified as "dynamic."

[0063] Figure 6 An exemplary image 600 of image data processed by means of a device according to an embodiment is shown. In this case, the device corresponds to or is similar to Figure 1 or Figure 4 Except that the motion probability of the determination device of the device is used here to generate the exemplary image 600, the exemplary image 600 is equivalent to Figure 5 Example imaging of . Figure 6Only the potential for compensating for random noise to avoid false positive detections is shown. Almost all false detections in the sky and on buildings are suppressed, while detections of moving objects remain. Only in the lower right image region do more false detections remain. In this example, these false detections are attributed to distortion caused by the windshield, i.e., a systematic effect.

[0064] Figure 7 An exemplary image 700 of image data processed by means of a device according to an embodiment is shown. In this case, the device corresponds to or is similar to Figure 1 or Figure 4 In addition to generating the exemplary image 700, an implementation of this device is used herein to filter the motion probability of Otherwise, the exemplary imaging 700 is equivalent to Figure 6 Thus, Figure 7 Shows the filtered motion probability Among them, the weighted value W k This can be encoded by the color saturation of the exemplary image 700: the greater the saturation, the higher the weighting value, i.e., the longer the corresponding probability of motion has been observed up to this point. This allows reliable measurements to be distinguished from unreliable ones. Consequently, a high degree of confidence is achieved with respect to the second object X2 and the static real world.

[0065] Figure 8 An exemplary image 800 of image data processed by means of a device according to an embodiment is shown. In this case, the device corresponds to or is similar to Figure 1 or Figure 4 Except that the binary filtered motion probability of the binary signal generating unit 428 of the device is used here to generate the exemplary imaging 800, the exemplary imaging 800 is equivalent to Figure 7 Thus, Figure 8 An exemplary binary motion probability is shown which has almost no false positive detections and shows a further increased confidence level with respect to the third object X3.

[0066] If an embodiment includes an “and / or” connection between a first feature and a second feature, this should be interpreted as meaning that the embodiment has both the first and the second feature according to one embodiment and either only the first or only the second feature according to another embodiment.

Claims

1. A method (200) for determining a motion state of at least one object (X; X1, X2, X3) in an environment of a vehicle (100), wherein: The vehicle (100) has at least one vehicle camera (102) for providing image data (105) representing the surroundings, wherein the method (200) comprises the following steps: - reading in (210) a motion measure (117) and quality information (119) generated by processing the image data (105), wherein the motion measure (117) comprises successive measurement values ​​generated by using correspondences between pixels in successive images represented by the image data (105) identified by means of a correspondence algorithm for detecting moving object pixels, and wherein the quality information (119) indicates the quality of the correspondence at least with respect to interference influences caused by noise; - generating (220) pixel-specific quality information (423) using the read-in quality information (119), wherein the pixel-specific quality information (423) indicates the quality of the correspondence for each pixel; and - using the motion measure (117) and the pixel-specific quality information (423) for each pixel, determining (230) a motion probability (P k ), wherein the motion probability (P k ) displays, for each pixel, the probability that the pixel belongs to a moving object (X; X1, X2, X3) or a static object (X), as a motion state.

2. The method (200) according to claim 1, characterized in that Having the motion probability (P k ) performing a temporal filtering step (240) to generate a filtered motion probability indicating the motion state 3. The method (200) according to claim 2, characterized in that In the implementation step (240), for each filtered motion probability Determine a confidence value, wherein the confidence value indicates that the filtered motion probability How reliably and / or accurately the motion state is described.

4. The method (200) according to claim 2 or 3, characterized in that In the implementation step (240), the filtered motion probability is implemented using a threshold comparison. , wherein the binarized filtered motion probability displays the motion state as static or dynamic.

5. The method (200) according to any one of claims 2 to 4, characterized in that In the implementation step (240), using a recursive filter and / or T memory element (T, T n ) is performed in the case of temporal filtering.

6. The method according to any one of claims 2 to 5, characterized in that In the implementation step (240), the filtered motion probability generated in at least one previous time step may be and the filtered motion probability generated at the current time step Fusion and / or accumulation, wherein the filtered motion probabilities generated in at least one previous time step are stored wherein the filtered motion probability generated in at least one previous time step is converted using the correspondence Transform to the current time step.

7. The method (200) according to claim 6, characterized in that When using the weighted value (w k ) to perform fusion and / or accumulation to generate filtered motion probabilities generated in at least one previous time step Weighted, wherein the filtered motion probability generated in at least one previous time step is accumulated according to how many previous time steps " to adjust the weighted value (w k ).

8. The method (200) according to any one of the preceding claims, characterized in that Error propagation methods and / or machine-learned models are used in the generating step (220).

9. A method (300) for operating a vehicle system (150) of a vehicle (100), wherein: The method (300) comprises the following steps: - determining (310) a motion state of at least one object (X; X1, X2, X3) in the environment of the vehicle (100) by means of a method (200) according to any one of the preceding claims; and - using the movement state, generating (320) a control signal (135) for output to the vehicle system (150) in order to actuate the vehicle system (150).

10. A device (120; 140) configured to implement and / or control the steps of the method (200; 300) according to any one of the preceding claims in a corresponding unit (130; 422, 424, 426, 428).

11. A computer program configured to carry out and / or control the steps of the method (200; 300) according to any one of the preceding claims.

12. A machine-readable storage medium in which the computer program according to claim 11 is stored.

Images