Method for detecting an object, and motor vehicle
The method employs curved search patterns from the vehicle to simplify pixel assignment across overlapping image sensors, enhancing real-time object detection accuracy and efficiency on moving vehicles.
Patent Information
- Application Number
- PCT/EP2025/051096
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2024-01-17
- Filing Date
- 2025-01-16
- Publication Date
- 2025-07-24
AI Technical Summary
Existing methods for object detection using multiple image sensors on a moving vehicle struggle with efficiently assigning pixels from overlapping image areas to recognize objects in real-time, often requiring complex grid calculations and synchronized image capture.
A method involving the use of curved search patterns emanating from the vehicle, assigning pixels along these curves to identify objects in overlapping image areas, reducing data volume by focusing on pixel groups aligned with these patterns, and incorporating time stamps and vehicle motion data for accurate assignment.
Enables reliable and precise real-time object detection by simplifying pixel assignment, reducing data complexity, and accounting for vehicle movement, even with asynchronous image capture.
Smart Images

Figure EP2025051096_24072025_PF_FP_ABST
Abstract
Description
[0001] DESCRIPTION
[0002] Method for detecting an object and motor vehicle
[0003] The invention relates to a method for detecting an object from a carrier moving on a ground surface. The carrier should, in particular, be a motor vehicle moving on the road or another ground surface. The method for detecting an object can be part of a method for the automated driving of a motor vehicle, in which, after the object has been detected, control commands are issued to actuator units of the motor vehicle that take the detection into account. It can also be part of a method for assisting a driver when driving a motor vehicle, wherein, here too, commands are issued to actuators of the motor vehicle depending on the detection of an object. In this case, the actuators can, in particular, be output devices for warning the driver and / or instructing the driver so that the driver can react appropriately.The invention also relates to a motor vehicle equipped accordingly.
[0004] It is intended that a plurality of image sensors capturing pixel images are located on the carrier / installed on it / formed on it. Such digital image sensors preferably capture pixel images in arrays in which the pixels are arranged in rows and columns, overall in a checkerboard-like grid pattern.
[0005] In the case of a motor vehicle as a carrier moving on the ground, it is typically the case that five cameras are provided, four of which are oriented in directions 90° apart from one another, for example forwards, left, backwards, and right. (The value of 90° refers to a central directional axis of the respective cameras, which are preferably designed as optical cameras, whereby a small deviation of up to 5° is possible (so that the specification includes an angular range of 85° to 95°)). These cameras preferably each have an image field of at least 150°, preferably of at least 160°, more preferably of at least 170°, and particularly preferably of at least 180°.
[0006] The fifth camera is preferably also directed forward, but is located at a different height within the vehicle: While the other four cameras can be positioned at approximately the same height as the vehicle's occupants' seats or lower, the fifth camera can be located in the roof area of the vehicle and thus capture a wider field of view ahead. Typically, the cameras are arranged in such a way that there is a larger overlap in the imaging field of the individual cameras.
[0007] The question here is how the individual pixel images captured by the image sensors can be compared. It is often the case that image areas of pairs of image sensors overlap (in images captured simultaneously; the overlapping image areas can be seen particularly when the otherwise moving vehicle / motor vehicle is stationary; this thus constitutes a spatial overlap). In this case, an object in the overlapping image area should be reliably detectable by both images of the image sensor pair based on the pixel data (color values, gray values, etc.) contained in the individual pixel images.
[0008] It is known from US 2014 / 0055573 A1 that 3D data recorded by several cameras on a vehicle are projected onto a plane, followed by calibration to enable the images of two cameras to be superimposed into one image and to capture a three-dimensional object.
[0009] WO 2016 / 198059 A1 is concerned with the generation of a virtual image of a
[0010] Vehicle environment with camera surround view system (camera
[0011] Environment detection system). Here, sampling points are defined using a three-dimensional grid, which is preferably defined in the space surrounding the motor vehicle, i.e. not with respect to the vehicle itself. Such a grid arrangement complicates calculations. In the present case, it would be desirable if objects could be detected in real time, if possible, in particular using images recorded essentially simultaneously by the image sensors. This avoids disadvantages that arise, for example, in the method according to US 11 544 895 B2, according to which a forward-looking camera first records an image of an environment and then a rear-looking camera records the same environment after passing a point.
[0012] It is accordingly an object of the present invention to facilitate the assignment of pixels of different pixel images from several image sensors to one another in order to be able to recognize objects as well and clearly as possible in the case of overlapping image areas of pairs of image sensors.
[0013] To solve this problem, the invention proposes a method for detecting an object having the features according to claim 1, which comprises the steps:
[0014] - capturing at least one image each by the image sensors of at least one pair of image sensors with an overlapping image area;
[0015] - Assigning pixels of the image of a first image sensor of an image sensor pair with an overlapping image area to pixels of the image of a second image sensor of the same image sensor pair (in this way, image structures selected indirectly via the pixels, such as object edges, can be assigned to the corresponding image structures in the other image);
[0016] - Detecting an object based on assigned pixels. In the invention, pixel groups of the image are assigned based on curved search patterns, preferably emanating from the carrier in a plan view, and the assignment occurs along the curves. The curves together form a search pattern. This is, in a sense, placed / projected into the images, and the pixel groups affected by the curves of the search pattern can then be assigned to image structures in the overlap area, and thus to objects that are depicted in both images. The concept of curves in this case encompasses any type of curve, i.e., straight as well as curved curves, and possibly both simultaneously in a search pattern.The detection step can also be formulated as follows: detecting an object on the basis of pixels assigned in this way, wherein pixel groups of the image of the first image sensor and the image of the second image sensor are each assigned to a curve, preferably emanating from the carrier in plan view, and the assignment of pixels of the image of the first image sensor of the image sensor pair with overlapping image area to pixels of the image of the second image sensor of this image sensor pair is carried out on the basis of the curves.
[0017] Solving the problem also involves providing a motor vehicle with a control unit that implements this method.
[0018] The method deliberately avoids defining grids. In addition, the pixel groups are now assigned to the carrier (in the example, the moving motor vehicle), and preferably emanate from it. Since the image sensors are usually fixed to the carrier / motor vehicle, the spatial relationship between the image areas of the image sensors is also fixed, which facilitates the assignment of the pixels to one another. In an image from one image sensor, for example, an imaginary curve along certain pixels can always correspond to the same imaginary curve along certain pixels in the image area of the neighboring image sensor (the second image sensor in the image sensor pair) and capture a specific area of the environment. Detecting an object can involve detecting any type of structure, surface, or topographical information.Within the scope of the invention, the images are reduced to the search pattern or even just individual curves thereof, thus reducing the data volume used for object recognition, and object recognition is carried out on the basis of the images thus reduced.
[0019] The main application, particularly in the automated driving of a motor vehicle or when a vehicle assistance system is assisting a driver, is where an obstacle is detected as an object. This obstacle preferably originates from the ground surface on which the carrier is moving. The obstacle can extend upwards (another motor vehicle, guardrail, module housing) or downwards (pothole). The image sensors formed on the carrier are preferably located at a certain height above the ground surface. The light beam that travels from the obstacle to the image sensor is straight, which means that the obstacle located closest to the front determines the data value of the pixel (color value, gray value, etc.). Since the curves originate from the carrier and not necessarily from the image sensors, groups of pixels in the image are assigned to a curve.If you now imagine driving along such a pixel group, i.e., following the curve, in the image captured by the image sensor, the obstacle is typically the first object with a predetermined property. This predetermined property can include the gray value at the pixel being significantly different from the gray value of the surrounding area (a curb is light gray compared to dark gray asphalt); and the neighboring pixels of the respective pixel on the curve, based on which the object is to be recognized, can also be used to detect that it is the desired object (obstacle).
[0020] According to a first preferred embodiment of the invention, the curves emanating from the carrier in plan view are all straight radial rays emanating from a center point. The center point can be provided within the carrier, but also an imaginary point below the carrier (on the roadway below the motor vehicle), which, however, moves with the carrier. In plan view, the curves then still emanate from the carrier. Such radial rays are particularly easy to calculate and can have predetermined angular spacings (approximately from 0.2° to 2°, preferably approximately between 0.3° and 0.7°, approximately 0.5°). However, the radial rays do not have to have a uniform angular spacing. When using such straight radial rays, pixels of the images recorded by neighboring image sensors (i.e., those from image sensor pairs with an overlapping image area) can be particularly easily assigned to one another.
[0021] Alternatively, at least some of the curves can be curved. This allows objects to be detected that are obscured in an image from one image sensor but visible to the neighboring image sensor, so that from the perspective of the first image sensor, it is possible to "look around the corner."
[0022] According to a further alternative, additional pixels are defined by branching the curves emanating from the carrier in plan view, with the branches being provided beyond the carrier. This allows for the fact that the distances between the curves increase with distance from the carrier / motor vehicle to be taken into account, so that the branching ensures sufficient density of detection of objects even farther away from the carrier / motor vehicle. The branching can occur from a predetermined distance / within a predetermined distance range; there can be further branches at a later point with respect to one or both branches.
[0023] According to a preferred embodiment, the acquisition time is stored as a so-called time stamp when the images are captured. The time stamps are taken into account when assigning the pixels. This embodiment takes into account the fact that with overlapping image areas that are large enough that they still overlap even if the image sensors of image sensor pairs do not capture a respective image at exactly the same time, the situation arises that one image is captured at a first position of the wearer / motor vehicle, and a second image is captured at a second position of the wearer / motor vehicle that has since moved further. (The time interval can be, for example, between 50 and 200 ms.)) However, because the curves are assigned to the wearer / the vehicle, i.e. are defined in a world system of the wearer, the world system in the second situation (post-movement) can be calculated back to the world system in the first situation (at the time the first image was taken) or vice versa, which still enables the assignment, even if the image sensors cannot be perfectly synchronized.
[0024] This embodiment demonstrates a significant advantage of the inventive definition of the curves starting (in plan view) from the carrier / motor vehicle. Preferably, in addition to the time stamps, a speed of the carrier is also taken into account, alternatively or additionally, a steering angle of the wheels (measured steering angle of the motor vehicle, etc.) and / or a yaw angle of the carrier (measured by a corresponding sensor) is taken into account, thereby achieving a particularly high degree of accuracy in the assignment.
[0025] According to a preferred embodiment of the invention, in addition to the image sensors, a radar and / or LiDAR device (light detection and ranging or light imaging, detection and ranging, loosely translated as "light-based object detection and distance measurement") is used, and its measurement results are used to assign the pixels of the images and / or to detect the object. In particular, a three-dimensional space is constructed and filled with the virtual curves / search pattern. This also ensures high precision and reliably detects an object.
[0026] The motor vehicle according to the invention comprises at least two digital sensors for obtaining data in a detection area, of which at least one is an image sensor and preferably both are image sensors for recording pixel images, wherein the at least two (image) sensors as a sensor pair have an overlapping detection area / image area when data / images are recorded simultaneously.The motor vehicle comprises a device for evaluating the data / images, wherein the device is designed to define at least one two-dimensional and preferably one three-dimensional virtual detection space / image space by pixel groups of the images / in the data, each of which is assigned to a curve (preferably emanating from the motor vehicle in plan view), and, based on the pixel groups (thus assigned to a curve) of the images of the (image) sensor pair with an overlapping detection area / image area, to assign pixels of the image of a first image sensor of the respective (image) sensor pair to pixels of the image / in the data of a second sensor of the respective (image) sensor pair, wherein the evaluation device is further designed to detect an object based on pixels thus assigned. The motor vehicle implements the method according to the invention, so that the above-mentioned advantages apply equally to it.
[0027] According to an advantageous embodiment, the motor vehicle is designed to drive automatically and its evaluation device is designed to detect at least obstacles as objects.
[0028] According to a further advantageous embodiment, the motor vehicle comprises, in a manner otherwise known, optical cameras as image sensors, which preferably comprise at least four cameras, each having an image area of at least 150°, wherein further preferably a first camera is directed forward and a fifth camera is provided, which is also directed forward but is provided at a different height of the vehicle than the first camera.
[0029] The invention can also be implemented as a motor vehicle with at least two digital sensors for recording images in a respective detection area, wherein at least one sensor provides a radar and / or LiDAR device and at least two sensors as a sensor pair have an overlapping detection area (spatial overlap) when images are recorded simultaneously, with a device for evaluating the images, wherein the device is designed to define at least one two-dimensional and preferably one three-dimensional virtual detection space by point groups of the images, each of which is assigned to a curve, preferably emanating from the motor vehicle in plan view,and to assign points of the image of a first sensor of the respective sensor pair to points of the image of a second sensor of the respective sensor pair based on the point groups (associated in this way or in this way with a curve) of the images of the sensor pair with overlapping detection range, wherein the device for evaluation is further designed to detect an object based on points thus assigned.
[0030] For use cases or application situations that may arise during the method and which are not explicitly described here, it may be provided that, in accordance with the method, an error message and / or a request to enter user feedback is issued and / or a default setting and / or a predetermined initial state is set.
[0031] The invention also includes the control device for the motor vehicle. The control device can have a data processing device or a processor device that is configured to carry out an embodiment of the method according to the invention. For this purpose, the processor device can have at least one microprocessor and / or at least one microcontroller and / or at least one ASIC (Application Specific Integrated Circuit) and / or at least one FPGA (Field Programmable Gate Array) and / or at least one DSP (Digital Signal Processor). In particular, a CPU (Central Processing Unit), a GPU (Graphical Processing Unit) or an NPU (Neural Processing Unit) can be used as the microprocessor. Furthermore, the processor device can have program code that is configured to carry out the embodiment of the method according to the invention when executed by the processor device.The program code can be stored in a data memory of the processor device. The processor device can be based, for example, on at least one circuit board and / or on at least one SoC (System on Chip).
[0032] The invention also includes further developments of the motor vehicle according to the invention that have features already described in connection with the further developments of the method according to the invention. For this reason, the corresponding further developments of the motor vehicle according to the invention are not described again here.
[0033] The motor vehicle according to the invention is preferably designed as a motor vehicle, in particular as a passenger car or truck, or as a passenger bus or motorcycle.
[0034] As a further solution, the invention also encompasses a computer-readable storage medium comprising program code which, when executed by a computer or computer network, causes the computer to carry out an embodiment of the method according to the invention. The storage medium can be provided at least partially as a non-volatile data memory (e.g. as a flash memory and / or as an SSD - solid state drive) and / or at least partially as a volatile data memory (e.g. as a RAM - random access memory). The storage medium can be arranged in the computer or computer network. However, the storage medium can also be operated on the Internet, for example, as a so-called app store server and / or cloud server. The computer or computer network can provide a processor circuit with, for example, at least one microprocessor.The program code may be provided as binary code and / or as assembly code and / or as source code of a programming language (e.g. C) and / or as a program script (e.g. Python).
[0035] The invention also encompasses combinations of the features of the described embodiments. The invention therefore also encompasses implementations that each comprise a combination of the features of several of the described embodiments, unless the embodiments are described as mutually exclusive.
[0036] Exemplary embodiments of the invention are described below. Shown are:
[0037] Fig. 1A shows a motor vehicle according to an embodiment of the invention in which the method according to an embodiment of the invention can be implemented,
[0038] Fig. 1B shows the structure of the data processing device of the motor vehicle from Fig. 1A;
[0039] Fig. 2 is a plan view of the motor vehicle of Fig. 1A in conjunction with the curves defined according to an embodiment of the invention and an object to be detected;
[0040] Fig. 3A shows an exemplary image taken by a first camera of the vehicle of Fig. 2, Fig. 3B shows an exemplary image taken by a second camera of the vehicle of Fig. 2;
[0041] Fig. 4 shows a first alternative course of curves in a first alternative embodiment of the method according to the invention; and
[0042] Fig. 5 shows a further alternative course of curves in a second alternative embodiment of the method according to the invention.
[0043] The exemplary embodiments explained below are preferred embodiments of the invention. In the exemplary embodiments, the described components of the embodiments each represent individual features of the invention that can be considered independently of one another, each of which also develops the invention independently of one another. Therefore, the disclosure is intended to encompass combinations of the features of the embodiments other than those shown. Furthermore, the described embodiments can also be supplemented by further features of the invention already described.
[0044] In the figures, the same reference symbols designate elements with the same function.
[0045] A motor vehicle shown in Fig. 1A and designated as a whole by 1 therein comprises a central data processing device 10 and five cameras, of which a front camera 12 records the front area of the motor vehicle, a second camera 14 (not shown in precise location here) records the right-hand area of the motor vehicle, a third rear-facing camera 16 records the area behind the motor vehicle 1 and a left-hand camera 18 records the area to the left of the motor vehicle, with a fifth camera 20 also recording the area in front of the motor vehicle, but from a higher vehicle height than camera 12, so that additional information can be obtained. The images from cameras 12, 14, 16, 18 and 20 are fed to data processing device 10, which, in its configuration as shown in Fig.1B comprises: a device 10-1 for receiving the images from the cameras, a device 10-2 for assigning the images to one another according to an assignment rule which carries out an assignment based on the curves explained below, a device 10-3 for detecting an object based on the assignment and a device 10-4 for controlling actuators of the motor vehicle 1 (for which purpose an individual actuator 20 is shown here as an example in Fig. 1A) depending on the detection of an object and / or also on the non-detection of an object or the non-detection of an object in a predetermined field of view.
[0046] Fig. 2 illustrates a top view of the motor vehicle 1 from Fig. 1A in conjunction with a radial pattern that is fixed with respect to the world of the motor vehicle. Camera 12 captures the image area extending from the left edge 12-LR to the right edge 12-RR, and camera 14 captures the image area extending from the left edge 14-LR to the right edge 14-RR. The two image areas clearly overlap.
[0047] Starting from a (virtual) center point M, here imagined to be under the motor vehicle, radial rays are intended to capture the ground in front of, next to, and behind the motor vehicle. These rays are straight curves, designated accordingly as lines L1, L2, L3, L4, L5, L6, etc., up to line L48. Lines L41 to L1 and L1 to L9 are at least partially located in the image field of camera 12. Lines L3 to L22 are at least partially located in the image field of camera 14. Accordingly, the image fields of the first camera 12 and the second camera 14 overlap in the area of lines L3 to L9. The radial rays together form a search pattern.
[0048] An exemplary image B12 (“photograph”), which was taken by means of the camera 12, is illustrated in Fig. 3A. A corresponding image B14, which was taken by means of the second camera 14, is illustrated as an example in Fig. 3B. In the image B12, each of the virtual lines in the image area, L41 to L48 and L1 to L9, is assigned a pixel group which traces such a line. These pixel groups are referred to here as PG12-41, corresponding to line L41, PG12-42 corresponding to line L42, etc., and with PG12-1 corresponding to line L1, PG12-2 corresponding to line L2, etc. Accordingly, image B14 shows pixel groups, of which the pixel groups PG14-7, PG14-8, PG14-9 and PG14-10 are shown as examples, which correspond to the lines L7, L8, L9 and L10.Since the cameras 12, 14, 16, 18 and 20 are stationary in relation to the motor vehicle 1 (to the body / vehicle structure), the pixel group PG12-7 in images such as image B12 from camera 12 corresponds to the pixel group PG14-7 in images such as image B14 from camera 14. The course of the pixel groups PG12-1, PG12-2, etc. in image B12 as well as the pixel groups PG14-7 etc. in image B14 is always the same in all images with regard to the respective image frame, whereby only the allocation of the pixels with the corresponding pixel data values differs.
[0049] Thus, pixel groups PG12-7 and PG14-7 each correspond to line L7, so that pixel groups PG12-7 and PG14-7 correspond equally to one another. Thus, object 2 captured in the images can be recognized. Starting from the vehicle imaginary at the lower edge of images B12 and B14, one can walk along pixel group P12-7 and recognize corner point 12E. Similarly, one can walk along pixel group PG14-7 according to the arrow in Fig. 3B and recognize its corner point 2E. Ideally, the pixel data values at image point 2E in Fig. 3A should be identical for similar cameras 12 and 14, or mappable to one another for different cameras according to a known mapping rule. The device 10-2 of the data processing device 10 of the vehicle 1 assigns the pixel groups PG12-7 and PG14-7 to each other and in particular the pixel data values at the shown points corner point 2E of the object 2.This now enables the detection of object 2 in device 10-3 of data processing device 10 and the corresponding control of actuators in motor vehicle 1 by device 10-4 of data processing device 10. For example, a steering angle of motor vehicle 1 to the right must be prevented so that object 2 is not hit. Furthermore, based on the detection of corner point 2E, the further course of object 2 to the left can be detected and it can be provided that motor vehicle 1 avoids object 2. For example, object 2 can simply represent a sidewalk, and the front edge with corner point 2E can represent its boundary (the curb). Within the scope of the invention, the images are reduced to the search pattern or just individual curves thereof, thus reducing the data volume, and object recognition is carried out using the thus reduced data.
[0050] Images B12 and B14 are assumed to have been taken at the same time. However, a slight temporal offset can also be taken into account. In this case, a corresponding time stamp should be recorded when the images are taken. Based on this information, preferably in conjunction with further information, such as the speed of the motor vehicle 1, the steering angle, and the yaw angle of the motor vehicle 1, the assignment can be made in the device 10-2. In this case, the curves of the search pattern are mathematically adjusted to account for the vehicle movement, so that the pixel groups—in the example, the pixel groups PG12-7 and PG14-7—actually belong to one another.
[0051] As an alternative to using the lines L1, L2, L3, it can be provided according to Fig. 4 that in the plan view starting from the motor vehicle 1, curves K1, K2, K3, K4 etc. are defined, which are partly straight, like the curves K1, K8 and K9, but partly curved, so that it is possible to better capture certain areas (“around the corner”).
[0052] Since, in the radial arrangement according to Fig. 2, the density of the pixels captured by the pixel groups decreases starting from the center point M, it can also be provided that the lines L1 etc. branch out in the direction away from the motor vehicle 1. This is shown in Fig. 5: As an example, the modification LT is shown for the line L1 in a modification of Fig. 2, whereby the line LT continues into the line L1 'a, and here the line L1 'b also branches off. The line L1 'a continues as line LTaa and also branches out into the line LTab. The line LTb in turn branches out into the lines LTba and LTbb. It would be possible for only one of the lines to branch out again according to the first branch. More than one double branch (triple branch, quadruple branch, etc.) is also possible.
[0053] The search pattern does not necessarily have to be centered in the area of the support / supporting structure / body of the motor vehicle. It would also be possible to define the same or similar search patterns as described here, fixed to the support / supporting structure / body of the motor vehicle, but outside it.
[0054] The acquisition of measurement data (in the context of image acquisition by image sensors) may also include the acquisition of meta-data (high-level information) about the measurement points, as well as transition information (e.g., covariances, classification information, object identification), if available, or even meta-data related to the sensor itself, such as boundaries of the image area.
[0055] Overall, the examples demonstrate how mapping of measurements with multiple cameras (or sensors in general) can be provided using a world-fixed radial sampling pattern.
Claims
PATENT CLAIMS 1 . A method for detecting an object (2) from a carrier (1) moving on a floor surface, on which a plurality of image sensors (12, 14, 16, 18, 20) recording pixel images are formed, wherein at least one pair of image sensors provides an overlapping image area for simultaneously recorded images, comprising the steps: capturing at least one image each by the image sensors (12, 14) of at least one pair of image sensors (12, 14) with an overlapping image area; Assigning pixels of the image (B12) of a first image sensor (12) of an image sensor pair with an overlapping image area to pixels of the image (B14) of a second image sensor (14) of the same image sensor pair; Detecting an object on the basis of assigned pixels, wherein pixel groups (PG12-7; PG14-7) of the image are assigned to a curve (L1, L2, L3, ..., L7, ..., L48), preferably emanating from the carrier (1) in plan view, and the assignment is carried out on the basis of the curves.
2. Method according to claim 1, characterized in that an obstacle is recognized as the object which originates from the ground surface on which the carrier is moved, which is the first object (2) with a predetermined property on at least one of the curves (L7).
3. Method according to claim 1 or 2, wherein the curves emanating from the carrier (1) in plan view are all straight radial rays (L1, L2, L3, ..., L48) emanating from a center point (M).
4. Method according to claim 1 or 2, wherein at least some of the curves (K2, K3, K4, ..., K11, K12) have a curvature.
5. Method according to claim 1 or 2, wherein, in plan view, further pixels of the pixel groups are defined by branching of the curves (LT) emanating from the carrier, the branches being provided beyond the carrier.
6. Method according to one of the preceding claims, characterized in that when the images are captured, the capture time is stored as a time stamp and when the pixels are assigned, the time stamps are taken into account, wherein preferably additionally a speed of the carrier (1) and / or a steering angle of wheels of the carrier (1) and / or a yaw angle of the carrier (1) and / or the 3D vehicle movement measurement of an inertial measuring unit is / are taken into account.
7. Method according to one of the preceding claims, characterized in that in addition to the image sensors, a radar and / or LiDAR device is used and the measurement results of which are used in assigning the pixels of the images and / or detecting the object.
8. A motor vehicle (1) with at least two digital image sensors (12, 14, 16, 18, 20) for recording pixel images, wherein at least two image sensors (12, 14) as an image sensor pair have an overlapping image area for simultaneously recorded images (B12, B14), with a device (10) for evaluating the images, wherein the device is designed to define at least one two-dimensional and preferably one three-dimensional virtual image space by pixel groups of the images, each of which is assigned to a curve (L1, L2, L3; K1, K2; LT, ...), preferably emanating from the motor vehicle (1) in plan view, and to assign pixels of the image of a first image sensor of the respective image sensor pair to pixels of the image of a second image sensor of the respective image sensor pair based on the pixel groups of the images of the image sensor pair with an overlapping image area, wherein the device (10) for evaluating is further designed to detect an object (2) on the basis of pixels (PG12-7; PG14-7) assigned in this way.
9. Motor vehicle (1) according to claim 8, which is designed to drive automatically and whose device (10) for evaluation is designed to recognize at least obstacles (2) as objects.
10. Motor vehicle (1) according to claim 8 or 9, with optical cameras as image sensors (12, 14, 16, 18, 20), which preferably comprise at least four cameras (12, 14, 16, 18), each having an image area of at least 150°, wherein further preferably a first camera (12) is directed forwards and a fifth camera (20) is provided, which is also directed forwards, but is provided at a different height of the vehicle (1) than the first camera (12).
11. A motor vehicle with at least two digital sensors (12, 14, 16, 18, 20) for recording images in a respective detection area, wherein at least one sensor provides a radar and / or LiDAR device and at least two sensors (12, 14) as a sensor pair have an overlapping detection area for simultaneously recorded images (B12, B14), with a device (10) for evaluating the images, wherein the device is designed to define at least one two-dimensional and preferably one three-dimensional virtual detection space by point groups of the images, each of which corresponds to a curve (L1, L2, L3; K1, K2; L1', ...) preferably emanating from the motor vehicle (1) in plan view.) are assigned, and to assign points of the image of a first sensor of the respective sensor pair to points of the image of a second sensor of the respective sensor pair on the basis of the point groups of the images of the sensor pair with an overlapping detection range, wherein the device (10) for evaluation is further designed to detect an object (2) on the basis of points assigned in this way.
Citation Information
Patent Citations
Surround view generation
US11544895B2
Device and method for detecting a three-dimensional object using a plurality of cameras
US20140055573A1
Method for generating a virtual image of vehicle surroundings
WO2016198059A1
Apparatus for synthesizing three-dimensional images to visualize surroundings of vehicle and method thereof
US20140125774A1
Vehicle and control method thereof
US20160159281A1