Method and system for detecting an object

EP4706009A1Pending Publication Date: 2026-03-11AUMOVIO AUTONOMOUS MOBILITY GERMANY GMBH +1
View PDF 0 Cites 0 Cited by

Patent Information

Authority / Receiving Office
EP · EP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2024-06-17
Publication Date
2026-03-11

AI Technical Summary

Technical Problem

Existing methods for detecting objects in vehicle surroundings, especially small objects on the road, face challenges due to noise and contrast issues in image information, particularly in poor lighting conditions, leading to inaccurate detection results for autonomous driving.

Method used

A method that adjusts and merges consecutive image information to align pixels of the same object areas, using techniques like shifting, scaling, and rotating, and determining pixel correspondences to create aggregated image information with reduced noise and maintained contrast, enabling effective object recognition.

Benefits of technology

This approach allows for accurate detection of small objects even under adverse conditions by compensating for vehicle and object movement, reducing noise and blurring, and improving the accuracy of object recognition for autonomous driving.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure EP2024066760_26122024_PF_FP_ABST
    Figure EP2024066760_26122024_PF_FP_ABST
Patent Text Reader

Abstract

The invention relates to a method for detecting an object (O) in image information by means of at least one computer unit (2) of a vehicle (1).
Need to check novelty before this filing date? Find Prior Art

Description

[0001] Method and system for detecting an object

[0002] The invention relates to a method and a system for detecting an object in a sequence of image information provided by imaging sensors of a vehicle.

[0003] It is generally known that imaging sensors can be used to capture the vehicle's surroundings. Examples of imaging sensors include mono cameras, stereo cameras, and radar and laser-based sensors (LIDAR).

[0004] Autonomous driving functions require a vehicle to be able to detect obstacles on the road surface in the vicinity of the vehicle. With existing object detection methods, it is difficult to detect small objects lying on the road, such as tire fragments, etc.

[0005] When detecting small objects in the image information provided by imaging sensors, the main problems are noise and contrast. This is especially true in low-light conditions. To reduce noise, the exposure time is often increased. However, this results in blurred images due to vehicle movement and / or the movement of the detected object.

[0006] The use of image information with a low signal-to-noise ratio or blurred image information in known object detection algorithms leads to inadequate detection results and thus to an environment model that is too inaccurate for autonomous driving functions. Based on this, the object of the invention is to provide a method for detecting an object in image information that enables sufficiently accurate object detection of small objects even under adverse visibility conditions.

[0007] This object is achieved by a method having the features of independent patent claim 1. Preferred embodiments are the subject of the dependent claims. A system for detecting an object in image information is the subject of independent patent claim 15.

[0008] According to a first aspect, the invention relates to a method for detecting an object in image information in a vehicle. The vehicle has a sensor system for detecting and a computer unit for processing the image information.

[0009] The procedure includes the following steps:

[0010] First, several chronologically consecutive pieces of image information are received by the vehicle's sensors. In other words, a sequence of chronologically acquired image information is received. The image information contains information regarding at least one object in the vehicle's surrounding area.

[0011] The image information captured sequentially is then processed by the computer unit.

[0012] The received image information can be at least partially adjusted relative to one another, so that the pixels of the respective received image information that relate to the same sub-regions of the object are each located at the same local position in the image information. In other words, during processing, at least parts of the image information are changed in their position and / or orientation such that the object regions depicted in the image information are congruent or substantially congruent in the respective image information. By adjusting the received image information in this way, a set of adjusted image information is generated.

[0013] Alternatively or additionally, processing the image information may include determining pixel correspondences between partial image information contained in the multiple pieces of image information and relating to the same sub-regions of the object. In other words, it is determined pixel by pixel whether a pixel represented in the first piece of image information that relates to a specific sub-region of an object corresponds to a pixel in the second piece of image information, i.e., whether it represents the same sub-region of the object. The corresponding pixels may have a different position in the digital image due to the relative movement between the vehicle and the object.

[0014] Subsequently, the adjusted image information (i.e., the set of adjusted image information) or the partial image information that exhibits pixel correspondences is merged pixel by pixel. This creates aggregated image information. Pixel-by-pixel merging involves collecting and processing the pixel information of the respective pixels that are located at the same spatial position in the adjusted image information or that exhibit pixel correspondences, each to create a pixel-by-pixel overall piece of information. In other words, for example, an overall piece of information is calculated from the color and / or brightness information of the respective pixels that are located at the same position in the adjusted image information (corresponding xy position for 2D images or corresponding xyz position for 3D images), and this overall information forms the color and / or brightness information of the corresponding pixel of the aggregated image information.In the event that the determined pixel correspondences are used for merging, an overall information is calculated from the color and / or brightness information of the pixels contained in image information acquired at different times and at which pixel correspondences were determined, and this overall information forms the color and / or brightness information of a pixel of the aggregated image information.

[0015] Finally, the position of at least one object in the aggregated image information is detected. For this purpose, an object recognition algorithm can be used, for example. Preferably, in addition to the position, the object's dimensions and / or its height (i.e., the height measured relative to the road surface) are also determined.

[0016] The technical advantage of the proposed method is that, despite the movement of the vehicle and / or the object, it is possible to combine several pieces of image information acquired at different times to form an overall image in which the noise effects are reduced compared to the individual pieces of image information and in which the image contrast is maintained, since blurring of the object edges is at least largely avoided in the inventive combining of the image information.

[0017] According to one exemplary embodiment, the at least partial adaptation of the received image information relative to one another comprises shifting, scaling, and / or rotating the image information about at least one image axis. In other words, either parts of the image information or the entire image are shifted, scaled, and / or rotated such that at least the object-relevant image regions in the set of adapted image information are positioned precisely relative to one another and are thus aggregated precisely when the adapted image information is merged. This compensates for the image changes resulting from the relative movement between the vehicle and the detected object, thus enabling the adapted image information to be superimposed without the resulting aggregated image information being blurred.

[0018] According to one embodiment, pixel correspondences between pieces of image information are determined using a pixel matching algorithm. This algorithm is designed to detect which pixels in the sequence of received image information relate to the same object regions. These relationships determined by the pixel matching algorithm can be used when merging the image information, aggregating information from pixels that relate to the same object regions.

[0019] According to one embodiment, pixel correspondences between pieces of image information are determined using an artificial neural network. This artificial neural network is trained to detect pixels in the sequence of received image information that refer to the same object regions. Alternatively, pixel correspondences can also be determined using a recurrent network, a hyper-permutation network, or a transformer network.

[0020] According to one embodiment, the vehicle comprises a vehicle motion estimator. The output information of the vehicle motion estimator is used to at least partially adapt the received image information and / or to determine pixel correspondences between the received image information. The vehicle motion estimator provides, for example, output information in the form of the vehicle's future direction of movement, the future roll angle, the future pitch angle, and the future yaw angle. In other words, the vehicle motion estimator can predict which position the vehicle will assume at a future point in time. This information can advantageously be used to adapt the received image information and / or to determine pixel correspondences.

[0021] According to one embodiment, the vehicle comprises at least one control unit that provides object movement information containing details about the movement of the at least one object. In other words, the control unit provides output information that indicates or can be derived from where an object will be located in the future. The object movement information can be used to at least partially adapt the received image information and / or to determine pixel correspondences between the received image information. Object movement information can be used to determine the position in the image information at which the moving object will be located in the future, which is advantageous for adapting the received image information and / or for determining pixel correspondences between the received image information.

[0022] According to one embodiment, the method is carried out recursively in successive cycles, wherein a position of an object in the aggregated image information detected in a previous cycle is used in a subsequent cycle to define relevant partial image information with respect to which the temporally successive image information is to be adjusted or for which pixel correspondences are to be determined. This can significantly reduce the computational effort for adjusting the sequence of image information and / or for determining pixel correspondences between the image information. According to one embodiment, the image information is three-dimensional (3D) image information. The at least partial adjustment of the received image information and / or the determination of the pixel correspondences between the received image information takes place along three dimensions in space.Thus, the proposed method makes it possible to improve object detection in three-dimensional image information.

[0023] According to one embodiment, the direction of movement and / or the speed of movement of the object is determined based on motion blur artifacts resulting from the movement of the object to be detected in the temporally successive image information. The direction of movement and / or the speed of movement of the object is then used to at least partially adapt the received image information and / or to determine the pixel correspondences between the received image information. As a result, the direction of movement of the object can be derived from the sequence of received image information itself and used for image processing to avoid or reduce image blur resulting from the movement of the object.

[0024] According to one embodiment, the at least partial adjustment of the received image information relative to one another takes place recursively in several consecutive sub-steps. The adjustment of the image information thus takes place step by step, with partial adjustments of the image information being made in each sub-step, which, when performed consecutively, result in a final adjustment of the image information. For example, a partial shift, a partial scaling, or a partial rotation of the image information can be performed in each sub-step. This can improve the final adjustment of the received image information.

[0025] According to one exemplary embodiment, the pixel-by-pixel merging of the adjusted image information or the partial image information and / or the pixel-by-pixel merging of the partial image information that has pixel correspondences comprises at least summing the pixel-related brightness information and / or color information. Due to the at least partial adaptation of the image information and / or the use of the pixel correspondences, the brightness information and / or color information relating to object regions is thus amplified, whereas statistical noise effects are not subject to amplification because the image adaptation does not superimpose them congruently, or no pixel correspondences are determined for the noisy pixels. This leads to aggregated image information with reduced noise and high contrast despite the relative movement between the vehicle and the object to be detected.

[0026] According to one embodiment, the pixel-by-pixel merging of the adjusted image information or the partial image information and / or the pixel-by-pixel merging of the partial image information that has pixel correspondences comprises at least averaging of the pixel-related brightness information and / or color information. This makes it possible to achieve aggregated image information that is independent of the number of pieces of image information present in the sequence and thus merged.

[0027] According to one embodiment, the sensor system comprises at least one camera, at least one stereo camera, at least one radar sensor, and / or at least one LIDAR sensor. The image information can be provided by a single sensor. Alternatively, the image information can also be formed by merging images from multiple sensors or multiple different sensor types (e.g., a camera and a radar sensor).

[0028] According to one embodiment, the sensor system comprises at least one camera or a stereo camera. The at least partial adaptation of the received image information and / or the determination of pixel correspondences between partial image information is carried out taking into account calibration parameters of the at least one camera or stereo camera. This allows calibration inaccuracies that influence the sequence of image information to be compensated.

[0029] According to a further aspect, the invention relates to a system for detecting objects in image information using at least one computer unit of a vehicle. The vehicle comprises a sensor system for determining image information. The computer unit is configured to perform the following steps:

[0030] - receiving a plurality of temporally successive image information items from a sensor system of the vehicle, wherein the image information items contain information relating to at least one object in the surrounding area of ​​the vehicle;

[0031] - Processing the temporally successive image information, comprising: o at least partially adapting the received image information relative to one another, so that the pixels of the respective received image information relating to the same sub-regions of the object are each located at the same local position in the image information, thereby forming a set of adapted image information; and / or o determining pixel correspondences between partial image information contained in the plurality of image information items and relating to the same sub-regions of the object;

[0032] - pixel-by-pixel merging of the adapted image information or of the partial image information having pixel correspondences, thereby producing aggregated image information, wherein the pixel-by-pixel merging comprises collecting and processing the pixel information of the respective pixels which are located at the same local position in the adapted image information or which have pixel correspondences, in each case to form a pixel-by-pixel overall information;

[0033] - Detecting the position of at least one object in the aggregated image information.

[0034] "Image information" within the meaning of this disclosure refers to any information on the basis of which a representation of the vehicle's surroundings can be made. These are, in particular, digital, pixel-based images provided by imaging sensors, for example, a mono camera, a stereo camera, a radar sensor, or a lidar sensor.

[0035] The terms “approximately”, “essentially” or “about” mean, in the sense of the invention, deviations from the exact value by + / - 10%, preferably by + / - 5% and / or deviations in the form of changes that are insignificant for the function.

[0036] Further developments, advantages, and possible applications of the invention will become apparent from the following description of exemplary embodiments and from the figures. All described and / or illustrated features, individually or in any combination, are fundamentally part of the invention, regardless of their summary in the claims or their reference back to them. The content of the claims is also incorporated into the description.

[0037] The invention is explained in more detail below with reference to exemplary embodiments and the figures. They show:

[0038] Fig. 1 shows, by way of example, a schematic representation of a vehicle with a sensor system which is designed to detect objects in the surrounding area in front of the vehicle;

[0039] Fig. 2 shows an exemplary schematic representation of a system for detecting objects from a sequence of image information;

[0040] Fig. 3 shows an example of a schematic representation illustrating the detection of objects from a sequence of image information; and

[0041] Fig. 4 shows an example of a flow chart to illustrate the steps of a method for detecting an object in the surrounding area of ​​a vehicle.

[0042] Figure 1 shows, by way of example, a vehicle 1 equipped with a computer unit 2 and a sensor system 3 for sensing and detecting an object 0 in the surrounding area of ​​the vehicle 1. The sensor system 3 can comprise one or more sensors. Furthermore, the sensor system can comprise the same or different sensor types, wherein the sensor types can be, for example, a camera (in particular a mono camera), a stereo camera, a radar sensor and / or a LIDAR sensor. Furthermore, the vehicle 1 can have a vehicle's own motion estimator 4. The vehicle's own motion estimator 4 is designed to estimate the vehicle's own motion. The estimation of the vehicle's own motion can be based on information from an odometry unit of the vehicle 1. Alternatively or additionally, the vehicle's own motion can be determined by tracking images from a camera or other own motion estimation methods.

[0043] Fig. 2 shows a block diagram illustrating the structure of a system for detecting an object 0 based on an image sequence B1, ..., Bn recorded one after the other, where n is a natural number and n > 2.

[0044] First, several pieces of image information B1, ..., Bn recorded at different times are received. In other words, an image sequence is received that contains image information B1, ..., Bn recorded at different times. The image information B1, ..., Bn can, for example, be individual images from an imaging sensor and include pixel-related color and / or brightness information. This image information B1, ..., Bn can originate from a single sensor of the sensor system 3 or be formed from a fusion of information from several sensors. The sensors can have the same sensor type or different sensor types.

[0045] The received image information B1, ..., Bn is fed to an image processing unit 10.

[0046] According to a first embodiment, the image processing unit 10 is designed to at least partially adapt the received image information B1, ..., Bn, ie either an adaptation of the entire images relative to one another or a relative adaptation of partial information of the respective received image information B1, ..., Bn takes place. "Adaptation" here means a two-dimensional or three-dimensional position adjustment of the entire image information B1,

[0047] Bn or parts of this image information relative to one another, in such a way that after the adaptation the pixels of the respective image information B1, ..., Bn which relate to the same object areas each end up in the same position in the image information B1, ..., Bn. In other words, after the adaptation the same object areas of the object would end up congruent in the adapted image information. The adaptation of the image information can, for example, comprise a translational shifting of the image information B1, ..., Bn relative to one another, a scaling (i.e. an enlargement or reduction) of the image information B1, ..., Bn relative to one another, a rotation of the image information about one or more axes of a Cartesian image coordinate system (i.e. a change depending on the roll, pitch and / or yaw angle of the vehicle).This makes it possible to at least partially correct or reduce changes in the temporally successive image information that arise due to the relative movement of the vehicle and the object.

[0048] As an alternative to adapting the entire image information of the image information received one after the other, in a further exemplary embodiment of the invention, the received image information can also be adapted by the image processing unit 10 only with regard to partial image information. “Partial image information” is understood to mean information that relates only to a part of an image. For example, the image information received one after the other can be adapted in relation to an object, i.e. the areas of the received image information that each relate to an object are adapted relative to one another in such a way that the pixels of the sequence of partial image information relating to the respective object can be merged in the correct position. Here, too, the adaptation of the partial image information can involve a translational shifting of the partial image information relative to one another, a scaling (i.e.This can include enlarging or reducing the image information relative to one another, or rotating the image information about one or more axes of a Cartesian image coordinate system (i.e., changing the image information depending on the roll, pitch, and / or yaw angle of the vehicle). This can, for example, result in the image information of an object that becomes increasingly larger in the received image information as the vehicle approaches, being reduced in size such that the image information relating to the object in the adapted image information to be merged each have the same size.

[0049] The at least partial adjustment of the sequence of image information relative to each other can be performed in a single step. Alternatively, the at least partial adjustment can be performed iteratively in several sub-steps, i.e., a partial adjustment is performed in each sub-step until a final adjustment is achieved after several sub-steps.

[0050] In a further alternative embodiment, the image processing unit 10 can be configured to determine pixel correspondences in the received sequence of image information. The pixel correspondences indicate which pixels of the received image information relate to the same image content and / or object regions. For example, a pixel correspondence can relate to a corner of an object and indicate which pixels in the received image information relate to this respective corner of the object. The pixel correspondences can be determined for the entire received image information or merely for partial image regions, in particular for a partial image region that relates to an object contained in all received image information.

[0051] The image processing unit 10 may comprise an artificial neural network, in particular a convolutional neural network (CNN), a recurrent network, a hyper-permutation network or a transformer network for adapting the received image information, for adapting the image partial information and / or for determining the pixel correspondences between the received image information.

[0052] The system shown in Fig. 2 also comprises a merging unit 11. The adapted image information or pixel correspondences to the received image information provided by the image processing unit 10 are received by this merging unit 11.

[0053] The merging unit 11 is designed to merge and / or aggregate the adjusted image information, i.e. the image information which has been at least partially adjusted with regard to the position of the pixels relative to one another, on a pixel-by-pixel basis. This creates aggregated image information. “Merging and / or aggregating” is understood in particular to mean integrating or averaging the pixel-related color and / or brightness information. Alternatively, the “merging and / or aggregating” could be carried out by weighted summation with different weighting factors, for example by means of an exponential filter. For example, the weighting factors could be selected such that more recent pixel-related color and / or brightness information is given a higher weighting than pixel-related color and / or brightness information from further back in time.By merging the adjusted image information, in which the image changes caused by the relative movement of the vehicle to the surroundings have been compensated, the color and / or brightness information of pixels that represent, for example, an object area are added together, making these object areas more recognizable in the aggregated image information. In contrast, noise effects that are statistically distributed in the image information and therefore were not adjusted for position in the adjusted image information are not accumulated and are therefore not amplified in the aggregated image information.

[0054] In the event that the image processing unit 10 provides information on pixel correspondences between the image information B1, ..., Bn, the merging unit 11 can merge the color and / or brightness information of the pixels of the image information B1, ..., Bn for which pixel correspondences were determined. "Merging and / or aggregating" is also understood here to mean, in particular, integrating or averaging the pixel-related color and / or brightness information. Alternatively, the "merging and / or aggregating" could be performed by weighted summation with different weighting factors, for example, using an exponential filter. For example, the weighting factors could be selected such that more recent pixel-related color and / or brightness information is given a higher weighting than pixel-related color and / or brightness information from further back in time.This allows corresponding partial image information to be aggregated at the pixel level, resulting in aggregated image information. Since the image processing unit 10 detects pixel correspondences, in particular to objects or object regions that are contained in the temporally successive image information B1, ..., Bn, but at different positions, in different sizes, and possibly in different orientations due to the relative movement of the vehicle with respect to the object, by taking the pixel correspondences into account, those partial image information items that each relate to the same object regions can be combined at the pixel level. This makes these object regions more easily recognizable in the aggregated image information, and statistically distributed noise effects in the image information B1, ..., Bn are not amplified.

[0055] The aggregated image information provided by the merging unit 11 is received by a detection unit 12. The detection unit 12 is configured to detect one or more objects in the aggregated image information. For example, the detection unit 12 can implement an object recognition algorithm that detects and labels objects in the aggregated image information.

[0056] In addition, the detection unit 12 can be configured to estimate the position and size of the object. This estimate can be a two-dimensional or three-dimensional estimate in space. As a result of this estimate, for example, a bounding box can be generated that indicates the size and position of the object in the plane (2D) or in space (3D). The information on the position and / or size of an object can be used in the subsequent adaptation steps to determine with respect to which areas of the image information an adaptation is to be made, or for which areas of the image information, in particular, pixel correspondences are to be determined.

[0057] The detection unit 12 can also provide tracking information about an object, which indicates the future position the object will assume in the plane (2D) or in space (3D). This information can be used for at least partial adjustment of newly received image information. For this purpose, the tracking information can be transmitted to the image processing unit 10, so that the latter can perform the adjustment process based on the tracking information. This results in improved object-related image adjustment.

[0058] Fig. 3 schematically shows the generation of aggregated image information based on an exemplary image sequence of an environmental area of ​​a vehicle and the subsequent detection of objects based on the aggregated image information.

[0059] In this case, a sequence of image information B and a sequence of distance information T are received for this image information B. The image information B are, for example, digital images containing pixel-specific color and / or brightness information. Each digital image is assigned a set of distance information T, with the set of distance information T assigning depth information to each pixel of the digital image. This depth information indicates how far an environmental area represented by the pixel of the digital image is from a reference point of the sensor system or the vehicle.

[0060] After receiving the sequence of image information B and the sequence of distance information T, the image information is either partially adjusted or the pixel correspondences between the individual consecutive pieces of image information are determined. A corresponding adjustment is also performed for the distance information T, i.e., a translational shift, scaling, and / or rotation of the image information is applied in the same way to the associated distance information, so that the assignment of the distance information to the image information is retained even after the adjustment. In the event that pixel correspondences between the individual consecutive pieces of image information are determined, the corresponding pixel correspondences are also applied to the temporally consecutive pieces of distance information T.

[0061] Subsequently, as described above, the adjusted image information and the distance information associated with the image information are merged pixel by pixel. In particular, the color and / or brightness information associated with the respective pixels is added or integrated.

[0062] For example, the pixel-related color and / or brightness information can be averaged. This creates aggregated image information. Similarly, the pixel-related distance information received sequentially can be added or integrated, and preferably also averaged. This creates aggregated distance information.

[0063] By combining the aggregated image information and the aggregated distance information, an aggregated three-dimensional representation of the captured environmental area is created in which the objects are more easily recognizable, while the noise effects are suppressed.

[0064] Based on this aggregated three-dimensional representation, object detection can be performed. In particular, three-dimensional bounding boxes can be assigned to the objects, representing their size and position in space.

[0065] It is understood that the previously described based on Fig. 3

[0066] The process can also be performed with pure 2D image information, i.e., without distance information. The processed image information can originate from a single sensor or from a fusion of image information from multiple sensors and / or a fusion of image information from different sensor types.

[0067] The above-described method for detecting an object can be performed iteratively. After receiving multiple pieces of image information (e.g., in the form of n digital images, where n is a natural number), the image processing unit 10 first performs the described image processing, then combines the image information into aggregated image information, and then detects the object based on the aggregated image information. This process is preferably repeated after receiving another piece of image information, specifically in such a way that the oldest piece of image information from the set of image information processed in the previous iteration step is discarded and replaced by the newly received image information. Processing is thus preferably performed according to the so-called FIFO principle (first in - first out).

[0068] Fig. 4 shows a diagram illustrating the steps of a method for detecting an object based on a sequence of image information.

[0069] First, several temporally consecutive pieces of image information are received by a sensor system of the vehicle. The image information contains information regarding at least one object in the surrounding area of ​​the vehicle. The image information was recorded at different times, so that this image information exhibits differences due to the movement of the vehicle and / or an object depicted in the image information (S10). The temporally consecutive pieces of image information are then processed.

[0070] This processing may, on the one hand, comprise at least partial adaptation of the received image information relative to one another, so that the pixels of the respective received image information relating to the same sub-regions of the object are each located at the same local position in the image information, thereby forming a set of adapted image information.

[0071] Alternatively or additionally, the processing may comprise determining pixel correspondences between partial image information contained in the plurality of image information items and relating to the same partial areas of the object (S11).

[0072] Subsequently, the adjusted image information or the partial image information is merged pixel by pixel according to the determined pixel correspondences, resulting in aggregated image information. Pixel-by-pixel merging involves collecting and processing the pixel information of the respective pixels that are located at the same spatial position in the adjusted image information or that have pixel correspondences, each to form a pixel-by-pixel aggregate information (S12).

[0073] Finally, the position of at least one object in the aggregated image information is detected (S13). Preferably, in addition to the position, the extent of the object and / or the height of the object (i.e., the height measured relative to the road surface) are also determined.

[0074] The invention has been described above using exemplary embodiments. It is understood that numerous changes and modifications are possible without departing from the scope of protection defined by the patent claims.

[0075] 1 vehicle

[0076] 2 computer unit

[0077] 3 Sensor technology

[0078] 4 vehicle self-motion estimators

[0079] 5 Control unit

[0080] 10 Image processing unit

[0081] 11 Merging unit

[0082] 12 Detection unit

[0083] AB aggregated image information

[0084] B Image information

[0085] B1 , Bn Image information

[0086] 0 object

[0087] T Distance information

Claims

Patent claims 1 ) Method for detecting an object (0) in image information by means of at least one computer unit (2) of a vehicle (1 ), comprising the following steps: - receiving a plurality of temporally successive image information items (B1, , Bn) from a sensor system (3) of the vehicle (1), wherein the image information items (B1, ..., Bn) contain information relating to at least one object (0) in the surrounding area of the vehicle (1) (S10); - Processing the temporally successive image information (B1, ..., Bn), comprising: o at least partially adapting the received image information (B1, ..., Bn) relative to one another, so that the pixels of the respective received image information (B1, ..., Bn) that relate to the same sub-regions of the object (0) are each located at the same local position in the image information, thereby forming a set of adapted image information; and / or o determining pixel correspondences between partial image information contained in the plurality of image information items (B1, ..., Bn) and that relate to the same sub-regions of the object (S11); - pixel-wise merging of the adapted image information or the partial image information having pixel correspondences, thereby creating aggregated image information (AB), wherein the pixel-wise merging comprises collecting and processing the pixel information of the respective pixels that are located at the same local position in the adapted image information or which have pixel correspondences, each comprising a pixel-wise overall information (S12); - Detecting the position of at least one object (0) in the aggregated image information (S13). 2) Method according to claim 1, characterized in that the at least partial adaptation of the received image information (B1, ..., Bn) relative to one another comprises a shift, a scaling and / or a rotation of the image information (B1, ..., Bn) about at least one image axis. 3) Method according to claim 1 or 2, characterized in that the determination of pixel correspondences between partial image information is carried out by means of a pixel matching algorithm. 4) Method according to one of the preceding claims, characterized in that the determination of pixel correspondences between partial image information is carried out by means of an artificial neural network. 5) Method according to one of the preceding claims, characterized in that the vehicle (1) comprises a vehicle's own motion estimator (4) and that output information of the vehicle's own motion estimator (4) is used for at least partially adapting the received image information (B1, ..., Bn) and / or for determining pixel correspondences between the received image information (B1, ..., Bn). 6) Method according to one of the preceding claims, characterized in that the vehicle (1) comprises at least one control unit (5) which receives object movement information, the information on Movement of the at least one object (0), and that the object movement information is used to at least partially adapt the received image information (B1, Bn) and / or to determine pixel correspondences between the received image information (B1, Bn). 7) Method according to one of the preceding claims, characterized in that the method is carried out recursively in successive cycles, wherein a position of an object (0) in the aggregated image information (AB) detected in a previous cycle is used in a subsequent cycle to define relevant image partial information with respect to which the successive image information is to be adjusted or to which pixel correspondences are to be determined. 8) Method according to one of the preceding claims, characterized in that the image information (B1, ..., Bn) is 3D image information and that the at least partial adaptation of the received image information (B1, ..., Bn) and / or the determination of the pixel correspondences between the received image information (B1, ..., Bn) takes place along three dimensions in space. 9) Method according to one of the preceding claims, characterized in that based on the motion blur artifacts that arise due to the movement of the object (0) to be detected in the temporally successive image information (B1, ..., Bn), the direction of movement and / or the speed of movement of the object (0) is determined and that the direction of movement and / or the speed of movement of the object (0) is used to at least partially adapt the received image information (B1, ..., Bn) and / or to determine the Pixel correspondences between the received image information (B1, Bn) are used. 10) Method according to one of the preceding claims, characterized in that the at least partial adaptation of the received image information (B1, ..., Bn) relative to one another takes place recursively in several successive sub-steps. 11) Method according to one of the preceding claims, characterized in that the pixel-by-pixel merging of the adapted image information or the image partial information and / or the pixel-by-pixel merging of the image partial information which has pixel correspondences comprises at least a summation of the pixel-related brightness information and / or color information. 12) Method according to one of the preceding claims, characterized in that the pixel-by-pixel merging of the adapted image information or the partial image information and / or the pixel-by-pixel merging of the partial image information which has pixel correspondences comprises at least an averaging of the pixel-related brightness information and / or colour information. 13) Method according to one of the preceding claims, characterized in that the sensor system (3) comprises at least one camera, at least one stereo camera, at least one radar sensor and / or at least one LIDAR sensor. 14) Method according to one of the preceding claims, characterized in that the sensor system comprises at least one camera or a stereo camera and that the at least partial adaptation the received image information (B1, Bn) and / or the determination of pixel correspondences between partial image information taking into account calibration parameters of the at least one camera or stereo camera. 15) System for detecting objects in image information (B1, Bn) by means of at least one computer unit (2) of a vehicle (1), wherein the vehicle (1) comprises a sensor system (3) for determining image information, wherein the computer unit (2) is configured to perform the following steps: - receiving a plurality of temporally successive image information items (B1, ..., Bn) from a sensor system (3) of the vehicle (1), wherein the image information items (B1, ..., Bn) contain information relating to at least one object (0) in the surrounding area of the vehicle (1) (S10); - Processing the temporally successive image information (B1, ..., Bn), comprising: o at least partially adapting the received image information (B1, ..., Bn) relative to one another, so that the pixels of the respective received image information (B1, ..., Bn) that relate to the same sub-regions of the object (0) are each located at the same local position in the image information, thereby forming a set of adapted image information; and / or o determining pixel correspondences between partial image information contained in the plurality of image information items (B1, ..., Bn) and that relate to the same sub-regions of the object (S11); - Pixel-wise merging of the adjusted image information or pixel correspondences having partial image information, whereby aggregated image information (AB) is created, wherein the pixel-by-pixel merging comprises collecting and processing the pixel information of the respective pixels which are located at the same local position in the adapted image information or which have pixel correspondences, in each case to form a pixel-by-pixel overall information (S12); - Detecting the position of at least one object (0) in the aggregated image information (S13).