Method for determining approximate object position of dynamic object, computer program, device, and vehicle

Abstract

To provide a method for determining an approximate object position of a dynamic object, computer program, device and vehicle.SOLUTION: There is provided a method for determining an approximate object position of a dynamic object in surroundings of a vehicle. The vehicle includes two ultrasonic sensors and one on-vehicle camera. The method at least includes: capturing sensor data by the two ultrasonic sensors; determining a present reflection origin position of a static or dynamic object according to the sensor data; capturing at least one camera image by the on-vehicle camera; recognizing the dynamic object according to the camera image; and determining a present estimated position of the recognized dynamic object relative to the vehicle. When a position distance between the determined estimated position of the recognized dynamic object and the determined reflection origin position is less than or equal to a distance threshold value, an approximate object position of the dynamic object is determined according to the present reflection origin position as belonging to the recognized dynamic object according to the sensor data.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a method for determining an approximate object position of a dynamic object in the surroundings of a vehicle, the vehicle having two ultrasonic sensors and at least one camera. The present invention also relates to a computer program comprising instructions which, when executed by a computer, cause the computer to carry out the steps of the method according to the invention. Furthermore, the present invention relates to an apparatus for a vehicle, in particular a central or zone processing unit or a control device, comprising a processing unit adapted to carry out the steps of the method according to the invention. The present invention also relates to a vehicle comprising this apparatus. [Background technology]

[0002] A parking pilot system for a vehicle typically includes a number of ultrasonic sensors and an evaluation unit, which captures and processes the sensor data and issues a warning about possible obstacles. Additionally, the use of camera data or camera images in parking pilot systems is known, whereby, for example, static and / or dynamic objects in the vehicle's surroundings are recognized on the basis of image data by learned machine recognition methods and reprocessed, for example for emergency brake assistance, and / or a visualization of the vehicle's surroundings for the driver is generated by generating a surround view or a top-down view with an overlay or fade-in of additional information. For this purpose, the captured camera images can be rectified, processed and transformed, for example, by a central processing unit, a zone processing unit or a control device, among others.

[0003] The ultrasonic sensors of the parking pilot system are typically arranged in front and rear shock absorbers of the vehicle, respectively, over the entire vehicle width, in order to determine distance data between the vehicle and objects around the vehicle. The ultrasonic sensors are typically designed to emit ultrasonic signals. The emitted ultrasonic signals are reflected by obstacles. The reflected ultrasonic signals are received again by the ultrasonic sensors as echo signals, and from the propagation time, the distance to the obstacle is calculated. This distance determination is generally performed as a function of the propagation time of the direct and / or indirect echo signals or the direct and / or indirect reflected ultrasonic signals or reflected signals with respect to the emitted ultrasonic signals. This determination of the distance is typically performed by a sensor ASIC arranged in the housing of the ultrasonic sensor. The sensor data of the ultrasonic sensor thus represents or includes, for example, the reflected ultrasonic signals or the received echo signals, and / or the calculated distance, as well as optional characteristics of the received echo signals, in particular with respect to the emitted ultrasonic signals. The reflector position can be determined, typically according to the trilateration principle, depending on at least one directly received echo signal and at least one indirectly received echo signal and / or depending on at least two directly received echo signals or at least two indirectly received echo signals. The trilateration principle presupposes that the ultrasonic sensors adjacent to the emitting ultrasonic sensor, for example to the right and / or to the left, are designed to receive cross-echoes or received echo signals or indirectly reflected signals for the emitted ultrasonic signal. The determination of the reflector position is preferably performed by means of a processing device, for example by means of a microcontroller, for example arranged in the control device or in a zone or central processing device, which processing device, preferably a processor, advantageously captures and processes sensor data of a plurality, in particular of adjacent ultrasonic sensors.

[0004] Document DE102010045657A1 discloses a surroundings monitoring system for a vehicle, which comprises at least two distance sensors for distance recognition by propagation time measurement of detection signals.

[0005] Document DE102019205565A1 discloses a method for estimating the height of an object based on received ultrasound signals. Document DE102018216790A1 discloses a method for assessing the influence of objects in the surroundings of a vehicle on the driving maneuvering of said vehicle.

[0006] It is known that the position of a recognized object, for example a pedestrian, can be estimated based on at least one camera image, for example by means of a cardinal pointing of the object. This position estimation works for pedestrians, but not when, for example, the object is in the immediate vicinity of the on-board camera, since the object can no longer be completely captured in the camera image. Parking pilot systems use multiple on-board cameras with wide-angle lenses and can show the driver a surround view or a top-down view using a display device. These wide-angle cameras are essentially better suited to recognize nearby objects, but camera-based position estimation for objects in the vehicle's vicinity area can fail.

[0007] The Kalman filter is used to iteratively estimate the state parameters of the system based on the erroneous measurements, and the measurement errors are reduced. Applications are known in which radar signals or GNSS data are evaluated for the localization of moving objects. Here, there is a problem of relating the measurements to the trajectory of the moving object. That is, if the erroneous measurements are related to the trajectory of the object, relatively large errors occur in the determined position and further evaluation. For example, a typical parking pilot system often cannot distinguish between a pedestrian and an adjacent curb, which leads to inaccurate localization and movement estimation for the pedestrian. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] DE102010045657A1 [Patent Document 2] DE102019205565A1 [Patent Document 3] DE102018216790A1 Summary of the Invention [Problem to be solved by the invention]

[0009] The object of the invention is to improve the object position determination of dynamic objects in the vehicle's surroundings, in particular with regard to the continuous tracking of pedestrians as dynamic objects or the tracking of pedestrians in the vehicle's vicinity. [Means for solving the problem]

[0010] According to the invention, the above problem is solved according to independent claims 1, 12, 13 or 15. The present invention relates to a method for determining an approximately actual or approximate object position of a dynamic object in the surroundings of a vehicle. The vehicle has at least two ultrasonic sensors and at least one on-board camera. The method comprises a step of capturing sensor data with the at least two ultrasonic sensors of the vehicle. The capturing of the sensor data is in particular performed continuously. The sensor data is in particular representative of at least one captured echo signal of an emitted ultrasonic signal between the vehicle and an object in the surroundings of the vehicle, as well as preferably representative of a distance between the vehicle and the object, and optionally at least one characteristic of the received echo signal. The characteristic of the captured echo signal is in particular based on an evaluation of the captured echo signal and advantageously represents a probability for the presence of a dynamic object. The evaluation of the echo signal to determine the probability for the presence of a dynamic object is a preferred optional method step. A current reflector position of a static or dynamic object is then determined at least depending on the captured sensor data of the at least two ultrasonic sensors of the vehicle, in particular by a trilateration method. The determination of the current reflector position is in particular performed continuously based on the currently captured sensor data of the at least two ultrasonic sensors of the vehicle. In a further method step, at least one camera image is captured by the vehicle-mounted camera, in particular a continuous capture of the camera image is performed by the vehicle-mounted camera. Accordingly, preferably a series of camera images are captured by the vehicle-mounted camera. The dynamic object is then recognized according to the at least one captured camera image by at least one (first) trained machine recognition method, the trained machine recognition method being preferably a trained neural network. For example, the recognized dynamic object is another moving or non-moving vehicle, a standing or walking pedestrian, a cyclist or an animal, such as a dog or a cat. Then, according to the at least one captured camera image, a determination of a current estimated position of the recognized object relative to the vehicle is performed. It can be provided that this estimated position is additionally determined by a motion equation, for example, if the pedestrian as a dynamic object is at least partially covered by the current image and therefore can no longer be recognized in the image.Alternatively or additionally, the determined current estimated position can be determined by a base point determination or by another learned machine recognition method, in particular a trained neural network, and / or depending on a movement estimation of the recognized object. In particular, since the dynamic object is mobile, the current estimated position changes continuously over time. The movement estimation of the recognized object can advantageously be based on a change in the size of the object box for the recognized object and on a base point determination of the recognized object in the camera images captured sequentially in time. If in a subsequent method step it is determined that the position distance between the current estimated position of the recognized dynamic object in the surroundings of the vehicle, in particular relative to the vehicle, and the current reflector position is less than or equal to a distance threshold, the current reflector position is classified as belonging to the recognized dynamic object depending on the sensor data on which the determination of the current reflector position is based and / or depending on at least one characteristic of the echo signal, in particular, which characteristic of the echo signal can be determined optionally depending on the sensor data or the echo signal. The classification is preferably performed as a respective characteristic of the captured echo signals of the captured sensor data, depending on the amplitude of the captured echo signals underlying the sensor data and / or depending on the correlation coefficient between the captured echo signals and the emitted ultrasonic signals. When the processing device of the ultrasonic sensor determines a characteristic depending on the echo signals, in particular the emitted ultrasonic signals, and this at least one determined characteristic is provided in the sensor data, the sensor data can contain or represent this characteristic. Alternatively or additionally, in this method, at least one characteristic of the echo signal can also be determined depending on the captured sensor data. Preferably, a second trained machine recognition method, in particular a second neural network, is used to recognize whether the reflector position can be classified as belonging to a dynamic object based on the at least two characteristics of the echo signal. Then, depending on the reflector position classified as belonging to a dynamic object, an approximate object position of the dynamic object is determined, in particular by means of a Kalman filter. This determination of the approximate object position of the dynamic object is performed in particular continuously.The present invention provides the advantage that the accurate approximate object position of a dynamic object recognized on the basis of a camera image in the vicinity of the vehicle can be determined as a function of ultrasonic measurements. Ultrasonic sensors provide very accurate measurements in the vicinity of the vehicle, but the camera image, particularly in the vicinity of the vehicle, is in part not easily assessable. Furthermore, the assignment or association of the determined reflector position to the dynamic object is improved, since measurements on static objects, such as curbs, plants or pillars, are not taken into account. It is also particularly advantageous to be able to very accurately determine the approximate object position of a pedestrian in the vicinity of the vehicle, i.e. within, for example, 2 meters, which is important for parking maneuvers in parking lots, for example. This is because the camera image cannot completely capture the pedestrian, especially the legs and feet, so that the pedestrian position cannot be accurately estimated on the basis of the camera.

[0011] In an optional form of the invention, the classification of the current reflector position as belonging to a recognized dynamic object is additionally performed depending on the underlying sensor data of reflector positions classified as belonging to a recognized dynamic object in the surroundings of the current reflector position during a predetermined time interval prior to the current time point, where the predetermined time interval is, for example, in the range of 10 ms to 3 s. In other words, the classification takes into account whether the underlying sensor data of reflector positions classified as belonging to a recognized dynamic object in the surroundings of the current reflector position suggests a probability that the current reflector position belongs to a dynamic object. This classification can advantageously be performed by a further trained machine recognition method. Alternatively or additionally, the current reflector position can be assigned to a reflector position previously classified as belonging to a recognized dynamic object, in particular based on similar characteristics of the echo signals of the sensor data. Alternatively or additionally, the current reflector position may be classified or not classified as belonging to a recognized dynamic object, in particular based on the likelihood that the location of the reflector position previously classified as belonging to a recognized dynamic object changes in time. Since information of determined reflector positions classified as belonging to a dynamic object from the surroundings of the determined current reflector position is taken into account, the advantage arises that the current reflector position can be classified more accurately as belonging to a recognized dynamic object. In particular, the current reflector position determined in this manner can be better classified as belonging to a static object or to another dynamic object, i.e. as not belonging to a recognized dynamic object. This manner therefore ensures a more reliable association of the reflector position with the dynamic object, which leads to a more accurate determination of the approximate object position of the dynamic object.

[0012] In one development of the aforementioned optional embodiment, the surroundings of the current reflector position include reflector positions that are assigned to or classified as belonging to a dynamic object, the distance to the current reflector position being equal to or less than a distance threshold. This distance threshold is adapted in particular depending on the speed of the vehicle. Alternatively or additionally, the surroundings of the current reflector position include at least one ultrasonic cluster assigned to a dynamic object, the ultrasonic cluster having in particular a reflector position classified as belonging to a dynamic object and / or a spread and / or a predefined geometric shape. Alternatively or additionally, the surroundings of the current reflector position include at least one grid cell of a grid of current reflector positions, the grid subdividing the surroundings of the vehicle. These types of surrounding descriptions of the current reflector position provide the advantage that reflector positions classified as belonging to a dynamic object are further filtered, so that for example different dynamic objects of the same type, for example different pedestrians, can be more easily distinguished from one another, which makes the determination of the approximate object positions of the individual dynamic objects more accurate.

[0013] In another optional development of the invention, a current object velocity of the dynamic object and / or a current object movement direction of the dynamic object is determined depending on the determined approximate object positions of the dynamic object at different times. This development has the advantage that the determination of the object velocity and / or object movement direction is performed accurately, since the approximate object positions are determined accurately.

[0014] In a further optional embodiment, the determination of the approximate object position of the dynamic object and / or the determination of the current object velocity of the dynamic object and / or the current object movement direction of the dynamic object, respectively, is not performed if the number of source positions classified as belonging to the dynamic object is below a predetermined confidence number. In this optional embodiment, a higher reliability of the determined approximate object position, the determined object velocity and the determined object movement direction is achieved.

[0015] Furthermore, it may be contemplated that a statistical uncertainty determination is performed depending on the captured sensor data, and / or the determined current reflector position, and / or the reflector position classified as belonging to a dynamic object, and / or the determined approximate object position.The distance threshold is then adapted depending on the determined statistical uncertainty.In this optional embodiment, the reliability and accuracy of the determined approximate object position, the determined object velocity, and the determined object movement direction are increased.

[0016] Preferably, the distance threshold is in the range of 0.1 meters to 5 meters, in particular the distance threshold is 0.5 meters, 1 meter, 1.5 meters or 3 meters, with the advantage that reflector positions representing other dynamic objects in the vehicle's surroundings are not erroneously classified by the method or algorithm as belonging to the recognized dynamic object, or multiple dynamic objects can be distinguished from one another.

[0017] Optionally, the distance threshold is adapted depending on the vehicle speed, which has the advantage that the increasing uncertainty of the determination of the current estimated position of the recognized object with increasing vehicle speed is taken into account, whereby at higher vehicle speeds the classification of the reflector positions as belonging to the recognized dynamic object is more reliable and thus the determination of the approximate object position of the dynamic object is more accurate depending on the reflector positions classified as belonging to the dynamic object.

[0018] In another optional embodiment of the invention, in an additional method step, a normalization of at least a part of the underlying sensor data for the determination of the current reflector position can be performed with respect to its amplitude based on the determined angular position of the current reflector position with respect to the capture range of the ultrasonic sensor. The classification of the current reflector position as belonging to the recognized dynamic object is performed in this embodiment as a function of the normalized underlying sensor data and an amplitude threshold, preferably reflector positions having a normalized amplitude equal to or less than the amplitude threshold are classified as belonging to the recognized dynamic object. This embodiment ensures the classification of each current reflector position.

[0019] Optionally, it may be provided that a correlation coefficient between at least a portion of the captured underlying sensor data of the reflector position and the sensor signal emitted by the ultrasonic sensor is additionally determined. In this embodiment, the classification of the current reflector position as belonging to the recognized dynamic object is performed according to the determined correlation coefficient and a correlation threshold, and preferably, reflector positions having a correlation coefficient equal to or less than the correlation threshold are classified as belonging to the recognized dynamic object. This embodiment allows the classification of the respective current reflector position as belonging to a dynamic object more reliably, especially when the classification is additionally performed according to the normalized underlying sensor data and an amplitude threshold.

[0020] In another optional form of the invention, the number of reflections of the sensor signal emitted by the ultrasonic sensor is determined as a function of at least part of the captured sensor data underlying the current reflector position. A classification of the current reflector position as belonging to the recognized dynamic object is then performed as a function of the determined number of reflections and as a function of a number threshold, and reflector positions having a number of reflections equal to or less than the number threshold are classified as belonging to the recognized dynamic object. This embodiment allows a very reliable classification of the respective current reflector position as belonging to a dynamic object, in particular if the classification is additionally performed as a function of the normalized underlying sensor data and as a function of the amplitude threshold, and / or in particular if the classification is additionally performed as a function of the determined correlation coefficient and as a function of the correlation threshold.

[0021] Preferably, in an optional embodiment of the present invention, it may be provided that the classification of the current reflector position as belonging to a recognized dynamic object is performed by a second trained machine recognition method.

[0022] Furthermore, the invention relates to a computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the steps of the method according to any one of claims 1 to 11.

[0023] The invention also relates to an apparatus for a vehicle, or a central or zone processing device, or a control device, comprising at least one first signal input designed to provide at least one first signal representative of captured sensor data from at least one ultrasonic sensor of the vehicle. The apparatus also comprises a second signal input designed to provide a second signal representative of a captured camera image of an on-board camera. The processing device, in particular the processor, of the apparatus is configured to execute the steps of the method according to the invention.

[0024] The apparatus further includes an optional signal output, which is designed to generate control signals for a display device, a braking mechanism, a steering mechanism, and / or a drive motor, depending on the method implemented by the processing device.

[0025] Furthermore, the invention relates to a vehicle comprising an apparatus according to the invention or a central processing unit according to the invention or a zone processing unit according to the invention or a control device according to the invention. Further advantages will be apparent from the following description of exemplary embodiments, with reference to the drawings, in which: [Brief description of the drawings]

[0026] [Figure 1] FIG. 2 shows a vehicle and dynamic objects in the vehicle's immediate surroundings. [Diagram 2] FIG. 2 shows the method flow as a block diagram. [Diagram 3] FIG. 2 is a diagram showing the statistical distribution of the amplitude of an echo signal. [Figure 4] FIG. 13 is a diagram showing a statistical distribution of correlation coefficients of echo signals. [Figure 5a] FIG. 2 shows determined reflector positions around the vehicle. [Figure 5b] FIG. 2 shows rear reflector positions around the vehicle. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0027] In Fig. 1, a vehicle 100 in a parking space of a parking lot and a dynamic object 1 in the immediate surroundings of the vehicle 100 are shown in a plan view from above. In this example, the dynamic object 1 is a pedestrian. The vehicle 100 has an ultrasonic sensor system 110 including four ultrasonic sensors 111 on the front and rear bumpers respectively. The vehicle 100 also comprises a camera system 120 including at least a rear on-board camera 121, which preferably has a wide-angle lens. The on-board camera 121 is designed to capture the surroundings 90 of the vehicle 100. The vehicle comprises a central processing unit 150. The central processing unit 150 is electrically connected to the at least one on-board camera 121 for data transmission of the camera images. Advantageously, it may be contemplated that the camera system 120 includes at least four on-board cameras 121, each located on another side of the vehicle and capturing different areas of the surroundings 90 from various viewpoints, so that the central processing unit 150 or the control device can generate, for example, a surrounding view (surround view) based on the camera images captured from the four on-board cameras. The central processing unit 150 is also electrically connected to the ultrasonic sensor system or ultrasonic sensors 111 for data transmission of the sensor data of the ultrasonic sensors 111. The sensor data received by the central processing unit 150 from each ultrasonic sensor 111 advantageously includes or represents a respective captured echo signal related to the emitted ultrasonic signal, and / or includes or represents the emitted ultrasonic signal, and / or includes or represents distance data between the vehicle 100 and the object 1 in the surroundings 90 of the vehicle 100, determined using the ASIC in each ultrasonic sensor based on the echo signal, and optionally includes or represents at least one characteristic of the echo signal. Furthermore, the vehicle includes a display device 160, a braking mechanism 161, a steering mechanism 162, and / or a drive motor 163, and the central control unit 150 is designed to control the display device 160, the braking mechanism 161, the steering mechanism 162, and / or the drive motor 163 according to the received or captured sensor data and / or the received or captured camera images. A pedestrian is captured as a dynamic object 1 within the capture range of the on-board camera 121.In time, the dynamic object is now passing the first vehicle parked next to it as the static object 50. The intention of the pedestrian as the dynamic object 1 may be to pass the vehicle 100 along the intended direction of movement 2, where the pedestrian passes the static objects 50 and 51, i.e. the first and second vehicles parked next to it. Within the immediate vicinity 91 of the surroundings 90 of the vehicle 100, the pedestrian as the dynamic object 1 is at least temporarily partially, and possibly temporarily completely, covered by the capture range of the on-board camera 121 during its movement. It may also be the case that within the immediate vicinity 91 of the vehicle 100, certain body parts of the pedestrian as the dynamic object 1, for example the head or the feet, are temporarily not recognizable. In this case, based on the camera images of the on-board camera 121, an accurate localization of the pedestrian as the dynamic object 1 is not possible, e.g. a base point determination cannot be performed in such a case. The ultrasonic sensors of the ultrasonic system 110 capture stationary objects 50 and 51, i.e., vehicles parked nearby, and a pedestrian as a dynamic object 1. This capture is performed even while the pedestrian, i.e., the dynamic object 1, is covered in the capture area of ​​the vehicle-mounted camera. In order to more accurately determine the object position, object speed, and object movement direction of the pedestrian as the dynamic object 1 even while the pedestrian is covered in the capture area, according to the present invention, the sensor data of the dynamic object 1 based on the capture of the ultrasonic sensors is classified, i.e., differentiated from the sensor data of the ultrasonic sensors for the stationary objects 50 and 51.

[0028] The flow of the method is shown diagrammatically in a block diagram in Fig. 2. In a method step 210, sensor data is captured with or received from at least two ultrasonic sensors of the vehicle. The sensor data respectively represent, for example, the distance between the object 1, 50, 51 in the surroundings 90 of the vehicle 100 and the vehicle 100, and advantageously the captured echo signal, and in particular at least one characteristic of the echo signal, for example the correlation coefficient between the captured echo signal and the emitted ultrasonic signal. Then, in a step 220, a current reflector position of the object 1, 50, 51 relative to the vehicle 100 is determined depending on the captured current sensor data. In a further method step 230, at least one camera image is captured with the on-board camera 121. In a subsequent optional step 221, it may be provided that at least a part of the sensor data, including the captured echo signal, is normalized in terms of amplitude based on the angular position of the determined current reflector position with respect to the capture range of the ultrasonic sensor. In a further optional step 222, a correlation coefficient is determined between at least a portion of the captured underlying sensor data of the reflector position and the sensor signal emitted by the ultrasonic sensor, or, if this correlation coefficient has already been determined in the ultrasonic ASIC or ultrasonic sensor processing device and is included or provided in the sensor data, is read from the underlying sensor data. Furthermore, in an optional step 223, depending on at least a portion of the captured underlying sensor data of the current reflector position, a number of reflections for the sensor signal emitted by the ultrasonic sensor can be determined, or, if this number of reflections for the sensor signal emitted by the ultrasonic sensor has already been determined in the ultrasonic ASIC or ultrasonic sensor processing device and is included or provided in the sensor data, is read from the underlying sensor data. The method also includes the capture 230 of at least one camera image by means of the vehicle-mounted camera 121. Then, depending on the at least one captured camera image, a recognition 240 of the dynamic object 1 is performed. The recognition of the dynamic object 1 is preferably performed by a trained machine recognition method, in particular a neural network.It may be provided that the trained machine recognition method is designed to distinguish between individual object classes of dynamic objects or to separate subclasses of dynamic objects. Then, in step 250, depending on at least one captured camera image, a current estimated position of the recognized dynamic object 1 relative to the vehicle 100 is determined, in particular the current estimated position is implemented as a base point determination of the recognized dynamic object 1. Optionally, before the check 260, in optional step 255, it may further be provided that a distance threshold is adapted depending on the vehicle speed of the vehicle 100. In a further method step 260, it is checked whether the position distance between the determined current estimated position of the recognized dynamic object 1 and the determined current reflector position is less than or equal to a distance threshold. The distance threshold is preferably in the range of 0.1 meters to 5 meters, in particular the distance threshold is 0.5 meters, 1 meter, 1.5 meters or 3 meters. If in method step 260 it is recognized that the position distance between the determined current estimated position of the recognized dynamic object 1 and the determined current reflector position is less than or equal to the distance threshold, this reflector position determined from time to time is classified in step 270 as belonging to the recognized dynamic object 1, in particular as belonging to a recognized specific individual object class or subclass of dynamic objects, depending on the sensor data on which the determination of the current reflector position is based and / or depending on optionally determined characteristics of the echo signals of the sensor data. The classification 270 of the current reflector position as belonging to the recognized dynamic object can optionally also be performed additionally depending on reflector positions that are in the surroundings of the current reflector position during a predetermined time interval prior to the current time point and that have already been classified as belonging to the recognized dynamic object, whereby the classification 270 is advantageously performed depending on the sensor data on which these reflector positions are based and / or depending on optionally determined characteristics of the echo signals of the sensor data on which these reflector positions are based. The predetermined time interval is for example in the range of 10 ms to 3 s. The surroundings of the current reflector position preferably include reflector positions assigned to dynamic objects whose distance to the current reflector position is less than or equal to the distance threshold.Alternatively or additionally, the surroundings of the current reflector position include at least one ultrasonic cluster assigned to a dynamic object. In particular, the ultrasonic cluster includes reflector positions classified as belonging to a dynamic object. Alternatively or additionally, the surroundings of the current reflector position include at least one grid cell in which the current reflector position is located or assigned, the grid of grid cells subdividing the surroundings 90 of the vehicle 100 into rectangular or square grid cells or surrounding sub-areas, in particular uniformly distributed, in particular as a model. Preferably, the classification 270 of the current reflector position as belonging to a recognized dynamic object is performed depending on the normalized underlying sensor data determined in step 221 and depending on an amplitude threshold, preferably reflector positions having a normalized amplitude below the amplitude threshold are classified as belonging to a recognized dynamic object. Alternatively or additionally, the classification 270 of the current reflector position as belonging to a recognized dynamic object is performed depending on a correlation coefficient determined or provided in the sensor data and depending on a correlation threshold, preferably reflector positions having a correlation coefficient below the correlation threshold are classified as belonging to a recognized dynamic object. Furthermore, alternatively or additionally, it may be provided that the classification 270 of the current reflector position as belonging to the recognized dynamic object is performed depending on the determined number of reflections and depending on a number threshold, with reflector positions having a number of reflections below the number threshold being classified as belonging to the recognized dynamic object. Particularly preferably, the classification 270 of the current reflector position as belonging to the recognized dynamic object is performed by a second learned machine recognition method, in particular a trained second neural network. The trained second neural network can, for example, determine or recognize the probability that the current reflector position belongs to the recognized dynamic object based on one or more characteristics of the echo signal, for example at least one captured or normalized amplitude and / or the number of reflections obtained in the echo signal, and / or a correlation coefficient.After classification 270 of the current reflector position as belonging to the recognized dynamic object, a determination 280 of the approximate object position of the dynamic object is performed, in particular by a Kalman filter, depending on the reflector position classified as belonging to the dynamic object. Then, in an optional step 290, it may be intended to determine a current object velocity and / or a current object movement direction of the dynamic object depending on the determined approximate object positions of the dynamic object at different times. In one optional form, the determination 280 of the approximate object position of the dynamic object and / or the optional determination 290 of the current object velocity and / or the current object movement direction of the dynamic object are not performed if the number of reflector positions classified as belonging to the dynamic object is below a predetermined confidence number. Moreover, in at least one further optional step, a statistical uncertainty determination can be performed (not shown in FIG. 2) depending on the captured sensor data, the determined current reflector position, the reflector position classified as belonging to the dynamic object, and / or the determined approximate object position. Then, as an alternative or additionally, an adaptation 255 of the distance threshold is performed depending on the determined statistical uncertainty. The process is preferably carried out continuously.

[0029] 3 shows a schematic diagram of the statistical distribution of the amplitude 301 or amplitude amount of the echo signal reflected by the stationary objects 50 and 51 and the dynamic object 1. The graph 390 relates to the statistical distribution of the amplitude 301 or amplitude amount of the echo signal reflected by the stationary objects 50 and 51. The graph 380 relates to the statistical distribution of the amplitude 301 or amplitude amount of the echo signal reflected by the dynamic object 1. It can be seen that typically the dynamic object 1 does not have an amplitude 301 that is smaller than the first amplitude threshold 310 or larger than the second amplitude threshold 320. Therefore, if the amplitude of the echo signal is smaller than the first amplitude threshold 310 or larger than the second amplitude threshold 320, the presence of a dynamic object can be excluded with a high probability. However, if the amplitude is within a range 330 between the amplitude threshold 310 and the second amplitude threshold 320, i.e. above the first amplitude threshold 310 and below the second amplitude threshold 320, the classification is not clear, since within this range 330 there are both amplitudes of echo signals reflected by static objects and amplitudes of echo signals reflected by dynamic objects. If the range 330 is rather narrow, i.e. the amplitude of each echo signal is additionally normalized respectively according to the angular position of the reflector position determined for it within the capture range of the respective ultrasonic sensor, the exclusion of reflector positions caused by stationary objects is improved, since the amplitude of the echo signal also changes relatively significantly according to this angular position.

[0030] In FIG. 4, the statistical distribution of the correlation coefficient 401 between the emitted ultrasonic signal of the ultrasonic sensor 111 and the received echo signal of the ultrasonic sensor 111 is shown, where the reflection of the emitted ultrasonic signal occurs on static and dynamic objects. The graph 490 relates to the statistical distribution of the correlation coefficient 401 or the magnitude of the correlation coefficient 401 of the echo signal reflected on the static objects 50 and 51. The graph 480 relates to the statistical distribution of the correlation coefficient 401 or the magnitude of the correlation coefficient 401 of the echo signal reflected on the dynamic object 1. It can be seen that beyond the threshold value 410 for the correlation coefficient 401, there is no correlation coefficient that can be associated with a dynamic object. In other words, the correlation coefficient of the echo signal reflected on the dynamic object is typically below the threshold value 410 for the correlation coefficient 401. However, when the correlation coefficient is below the threshold value 410 for the correlation coefficient 401, within this range 420, there exists both the correlation coefficient 401 of the echo signal reflected on the static object and the correlation coefficient 401 of the echo signal reflected on the dynamic object, so that the classification as belonging to the dynamic object is not clear.

[0031] The classification 270 of the respective reflector location according to the characteristics shown in figures 3 and 4 of the underlying echo signal is preferably performed by a learned machine recognition method, in particular a trained neural network, which can take into account further characteristics of the underlying echo signal, for example the number of reflections identified in the echo signal, in order to assess the probability for the presence of a dynamic object. The evaluation of a large number of characteristics of the echo signal by a learned machine recognition method, in particular a trained neural network, makes the classification 270 of the current reflector location as belonging to the recognized dynamic object very reliable.

[0032] In Fig. 5a and Fig. 5b the determined reflector positions 501 in the surroundings 90 of the vehicle 100 are shown diagrammatically in the xy-plane, i.e. in a map viewed vertically from above, at different times t1 and t2, the reflector positions shown in Fig. 5a and Fig. 5b resulting for example from a situation in which a pedestrian as a dynamic object according to Fig. 1 passes behind the vehicle 100. The time t2 is for example after the time t1, but in another situation it could also be the other way around, for example when the pedestrian is moving in the opposite direction behind the vehicle. The determined reflector positions 501 thus represent a static object 50 in the area 510, a static object 51 (the vehicle parked next to Fig. 1) in the area 520 and a pedestrian as a dynamic object 1 in the area 530. Accordingly, the classification 270 of the respective current reflector position 501 in the method step 270 is preferably also performed depending on the reflector positions 501 in the surroundings of the current reflector position 501.

[0033] In Fig. 5b the situation is shown at a later time in Fig. 5a. In this exemplary embodiment, the area 510 representing the static object 50 is directly adjacent to the area 530 representing in this case a pedestrian as the dynamic object 1. The areas 510, 520 and 530 may also overlap (not shown here) if, for example, a pedestrian as the dynamic object 1 passes the vehicle 100 on a curbed walkway. As can be seen in Fig. 5b, the reflector positions in the map around the vehicle 100 are too close to each other in position to perform a reliable classification 270 of the (current) reflector position. The classification 270 is therefore performed depending on the sensor data on which the respective current reflector position 501 is based, which may include at least one characteristic of the echo signal, and / or depending on the determined characteristics of the echo signal of this sensor data and / or depending on the reflector positions 501 around the current reflector position 501, this classification 270 being particularly preferably performed by a trained machine recognition method as described above. [Explanation of symbols]

[0034] 1 Dynamic Objects 2. Intended direction of movement 50 Stationary objects 51 Stationary objects 90 perimeter 91 The immediate area around 90 100 vehicles 110 Sensor System 111 Ultrasonic Sensor 120 Camera System 121 In-vehicle camera 150 Central Processing Unit 160 Display device 161 Brake mechanism 162 Steering mechanism 163 Drive motor 210 Method Steps 221 Optional steps that follow 222 Further optional steps 223 Optional Steps 230 Further method steps / capturing at least one camera image 240 Dynamic Object Recognition 1 250 steps 255 optional steps 260 Checks / Further method steps 270 steps / classification of the current reflector position as belonging to a recognized dynamic object 280 Determining Approximate Object Position for Dynamic Objects 290 Optional step / optional determination of current object velocity and / or current object movement direction of a dynamic object 301 Amplitude of echo signals reflected by stationary objects 50 and 51 and dynamic object 1 310 First Amplitude Threshold 320 Second Amplitude Threshold 330 Range between the amplitude threshold 310 and the second amplitude threshold 320 380 Graph of the amplitude 301 or the statistical distribution of the amplitude quantity of the echo signal reflected by the dynamic object 1 390 Graph of the statistical distribution of the amplitude 301 or amplitude quantity of the echo signal reflected by stationary objects 50 and 51 401 Correlation coefficient of echo signals reflected by stationary objects 50 and 51 410 Threshold for correlation coefficient 401 420 Range in which the correlation coefficient is equal to or less than the threshold 410 for the correlation coefficient 401 480 A graph of the correlation coefficient 401 of the echo signal reflected by the dynamic object 1 or the statistical distribution of the magnitude of the correlation coefficient 401 490 A graph of the correlation coefficient 401 of the echo signals reflected by stationary objects 50 and 51 or the statistical distribution of the magnitude of the correlation coefficient 401 501 Reflection source position 510 areas 520 areas 530 Surroundings / Area

Claims

1. A method for determining the approximate object position of a dynamic object (1) in the vicinity (90) of a vehicle (100), wherein the vehicle (100) has at least two ultrasonic sensors (111) and at least one onboard camera (121), and the method steps are as follows: - The steps of capturing sensor data (210) using at least two ultrasonic sensors (111), - A step of determining (220) the current reflector position (501) of a static or dynamic object (1, 50, 51) according to the captured sensor data, - The steps of capturing at least one camera image (230) with the in-vehicle camera (121), - A step of recognizing (240) the dynamic object (1) in accordance with at least one captured camera image, - A step of determining (250) the current estimated position of the recognized dynamic object (1) relative to the vehicle (100) in accordance with at least one captured camera image, A method that includes at least the following: If the positional distance between the determined current estimated position of the recognized dynamic object (1) and the determined current reflector position (501) is less than or equal to the distance threshold, then the following steps are taken. - A step of classifying (270) the current reflector location as belonging to the recognized dynamic object (1) according to the sensor data that forms the basis for determining the current reflector location, using a learned machine recognition method, - A step of determining the approximate object position of the dynamic object (1) using a Kalman filter according to the position of the reflection source which has been classified as belonging to the dynamic object (1) (280), A method characterized by the implementation of [a certain action].

2. The method according to claim 1, wherein the classification (270) of the current reflector location (501) as belonging to the recognized dynamic object (1) is further performed in accordance with the underlying sensor data of the reflector location (501) that was classified as belonging to the recognized dynamic object during a predetermined time interval prior to the present time, around the current reflector location (501).

3. - The surrounding area (530) of the current reflector position (501) includes the reflector position (501) assigned to the dynamic object (1) such that the distance to the current reflector position (501) is less than or equal to a distance threshold, and / or - The surrounding area (530) of the current reflector location (501) includes at least one ultrasonic cluster assigned to the dynamic object (1), the ultrasonic cluster includes the reflector location (501) which is classified as belonging to the dynamic object (1), and / or - The surrounding area (530) of the current reflector location (501) has at least one grid cell to which the current reflector location (501) is located or assigned, and the grid of the grid cell subdivides the surrounding area (90) of the vehicle (100). The method according to claim 2.

4. The following steps - A step of determining (290) the current object velocity and / or current object direction of movement of the dynamic object (1) according to the determined approximate object position of the dynamic object (1) at different points in time. The method according to claim 1, wherein the method is further implemented.

5. The method according to claim 1, wherein the determination (280) of the approximate object position of the dynamic object (1), and / or the determination (290) of the current object velocity and / or the current object movement direction of the dynamic object (1), are not performed if the number of reflector positions (501) classified as belonging to the dynamic object (1) is less than a predetermined confidence number.

6. The following steps - A step of identifying statistical uncertainty based on the captured sensor data, the determined current reflector position (501), the reflector position (501) classified as belonging to the dynamic object (1), and / or the determined approximate object position, - A step of fitting the distance threshold (255) according to the identified statistical uncertainty, The method according to claim 1, wherein the method is further implemented.

7. The method according to claim 1, wherein the distance threshold is in the range of 0.1 meters to 5 meters.

8. The following steps - A step of normalizing (221) at least a portion of the sensor data that forms the basis for determining the current reflector position (501) with respect to its amplitude, based on the determined angular position of the current reflector position (501) with respect to the capture range of each of the ultrasonic sensors (111). Further measures were implemented, - The classification (270) of the current reflector position (501) as belonging to the recognized dynamic object (1) is performed according to the normalized underlying sensor data and at least one amplitude threshold. The method according to claim 1.

9. The following steps - A step of determining (222) the correlation coefficient between at least a portion of the sensor data that forms the basis of the captured reflector position and the sensor signals emitted by each of the ultrasonic sensors (111). Further measures were implemented, - The classification (270) of the current reflector location (501) as belonging to the recognized dynamic object (1) is performed according to the determined correlation coefficient and according to a threshold for the correlation coefficient. The method according to claim 1.

10. The following steps - A step of determining (223) the number of reflections of the sensor signal emitted by each of the ultrasonic sensors (111) in accordance with at least a portion of the sensor data that forms the basis of the captured current reflector position (501). Further measures were implemented, - The classification (270) of the current reflection source position (501) as belonging to the recognized dynamic object (1) is performed according to the determined number of reflections and the count threshold. The method according to claim 1.

11. The method according to claim 1, wherein the classification (270) of the current reflector location (501) as belonging to the recognized dynamic object (1) is performed by a second learned machine recognition method.

12. A computer program comprising instructions, wherein the instructions cause the computer to perform a step of the method according to any one of claims 1 to 11 when the computer program is executed by the computer.

13. A device for a vehicle (100), comprising the following components - A first signal input designed to provide at least one first signal representing sensor data captured from the ultrasonic sensor (111) of the vehicle (100), - A second signal input designed to provide a second signal representing the captured camera image of the in-vehicle camera (121) and It has at least the following features: - The apparatus is configured to perform a step of the method according to any one of claims 1 to 11, Device.

14. - Further including a signal output, the signal output is designed to generate control signals for a display device (160), a braking mechanism (161), a steering mechanism (162), and / or a drive motor (163) in accordance with the approximate object position of the dynamic object (1) determined by the method. The apparatus according to claim 13.

15. A vehicle (100) equipped with the device described in claim 13.

Images