Method and system for identifying interior configuration of vehicle

Abstract

The invention relates to a method (100) for detecting a configuration of an interior (2) of a vehicle (1). In the method, image data is acquired by means of at least one image acquisition device (3), which image data represents views of the interior (2) of the vehicle (1), each of which at least partially contains at least one stationary part (7, 8, 15) of the vehicle (1) and at least one movable component (9, 10, 11, 12, 13, 14) of the interior (2) of the vehicle (1). The image data is evaluated. In this case, the at least one stationary part (7, 8, 15) of the vehicle (1) is identified in order to assign the view to a corresponding region of the interior (2) of the vehicle (1). Furthermore, a configuration of the at least one movable component (9, 10, 11, 12, 13, 14) of the interior (2) is detected in a region of the interior (2) of the vehicle (1) that is associated with the view.

Description

Technical Field

[0001] The present invention relates to a method and a system for identifying the configuration of an interior space of a vehicle. In particular, the identification of the interior space configuration is based on image data acquired in the interior space of the vehicle by means of an image acquisition device, such as a camera. Background Art

[0002] In different situations, it may be necessary or desirable to determine the configuration of the interior in a vehicle. In particular, the seats of the vehicle may be adjusted, the seat backs may be adjusted backwards, or the tables or other movable parts may be unfolded or pulled out. With the implementation of Level 3 to Level 5 driver assistance functions (especially at Level 4 or at Level 5), other changes in the seat adjustment may also be made during driving, for example, the seats may be turned towards the interior of the vehicle. As a result, specific airbags must be triggered or inhibited depending on the configuration of the individual interior components. For example, if the front seats are turned, the front airbags must be disabled. If the seat backs are tilted backwards, for example, different airbags must be activated than if the seat backs are adjusted straight. The detection of these vehicle interior configurations generally requires sensors, such as position sensors (e.g. Hall sensors or potentiometers) in each movable interior component in order to detect the position of the interior components and the engagement of the components (e.g. when the seats are turned, when the backrests are folded, etc.).

[0003] The recognition of changes in the interior of a vehicle, for example also the recognition of seat occupancy, can be carried out based on a camera. In many different technical applications, the following goal is proposed: to analyze image data, that is, to analyze data representing an image or a sequence of multiple images (for example a video): whether an object is depicted in the image or the multiple images, and if necessary, which objects are depicted. It is often also of interest to recognize the movement or change of such objects based on these images or image data. For this purpose, the corresponding camera can be fixedly installed in the interior of the vehicle in order to capture a limited view of the interior. The corresponding image data can then be evaluated. The camera can be arranged, for example, in the dashboard, but there, parts of the interior may be blocked by parts of the vehicle or by vehicle occupants. However, a camera field that is visible and installed in the interior of the vehicle is undesirable on the user side. Summary of the invention

[0004] The object of the present invention is to further improve the detection of the configuration of the interior of a vehicle, in particular the detection should be carried out without the presence of dedicated sensor devices in the individual movable components in the interior of the vehicle.

[0005] This object is achieved according to the teaching of the independent claims. Different embodiments and further developments of the invention are the subject matter of the dependent claims.

[0006] A first aspect of the invention relates to a method, in particular a computer-implemented method, for identifying the configuration of the interior of a vehicle. In the method, image data are acquired by means of at least one image acquisition device, the image data representing a view of the interior of the vehicle, the views each at least partially containing at least one fixed part of the vehicle and at least one movable component of the interior of the vehicle. The image data are evaluated. In this case, the at least one fixed part of the vehicle is identified in order to assign the view to a corresponding area of ​​the interior of the vehicle. Furthermore, the configuration of the at least one movable component of the interior is identified in the area of ​​the interior of the vehicle to which the view is assigned.

[0007] The above method according to the first aspect is therefore based in particular on evaluating image data acquired by means of an image acquisition device, such as a camera. The image acquisition device records a view of the interior from which the arrangement of movable components in the interior can be recognized. By also recognizing the fixed parts of the vehicle, a reliable recognition of the arrangement of the movable components can be achieved. The image evaluation can be performed by means of the fixed parts of the vehicle.

[0008] The views recorded by the image acquisition device are used in the evaluation of the image data, wherein the configuration of the individual components can be recognized, for example, using machine learning or conventional image recognition methods. In this case, the method can use fixed and therefore unchanged parts of the vehicle and infer the configuration of one or more of the movable components. No special sensors (e.g. Hall sensors) are required in the interior or in the individual movable parts of the interior, such as in the seats. Since the method takes the fixed parts of the vehicle into account, no additional measurement markings are required, for example, on movable parts in the interior or at other locations in the interior of the vehicle.

[0009] The term "vehicle" as used herein relates in particular to passenger cars, including all types of motor vehicles, hybrid vehicles and battery-operated electric vehicles, and also vehicles such as vans, buses, trucks, delivery vehicles and the like.

[0010] The term "configuration of the interior of the vehicle" (with its equivalents "interior configuration of the vehicle", "configuration of the vehicle interior" or "vehicle interior configuration") as used herein is to be understood in particular to mean the arrangement of interior components, in particular including their position and orientation in space, i.e. in the vehicle coordinate system. This may in particular include their absolute position and / or orientation in space as well as their relative position and / or orientation with respect to other components of the vehicle or the vehicle interior. The term may encompass the configuration of a single component as well as the configuration of a plurality of components.

[0011] The term "image acquisition device" used here particularly refers to a camera, in particular a digital camera. The camera can capture still images (photos) or moving images (videos). The image acquisition device can capture or capture such images and output corresponding image data.

[0012] The term "view" as used here relates in particular to the field of view of an image capture device. In other words, the view shows a region of the vehicle interior that can be recorded with the aid of the image capture device. The view can, for example, change when the image capture device is moved. The view is assigned to a region of the vehicle interior, i.e. to a region visible in the view. The view can be two-dimensional or three-dimensional.

[0013] The term "fixed part (of the vehicle)" as used here refers in particular to a part of the vehicle that cannot be changed or moved. This can be, for example, a part of the vehicle body, such as the A-pillar or B-pillar of the vehicle. Parts of the interior can also be fixed, such as the center console or fixed structures or elements on the floor or interior.

[0014] The term "movable component (of the interior of the vehicle)" as used herein refers in particular to a component of the interior that can be moved in the interior, in particular by an operation or manipulation (manually or automatically), i.e., the position and / or orientation of the component in space is variable, in particular relative to fixed components of the vehicle. This may also include changes in the shape of the component.

[0015] Also as used herein, the terms "comprises," "comprising," "containing," "having," "having," "with," or any other variation thereof, shall cover a non-exclusive inclusion. Thus, for example, a method or apparatus that comprises or has a list of elements is not necessarily limited to those elements but may include other elements not expressly listed or inherent to such method or such apparatus.

[0016] In addition, unless expressly stated otherwise, "or" refers to an inclusive or, not an exclusive "or". For example, condition A or B satisfies one of the following conditions: A is true (or exists) and B is false (or does not exist), A is false (or does not exist) and B is true (or exists), and not only A but also B is true (or exists).

[0017] The terms "a" or "an" as used herein are defined as meaning "one or more." The terms "an additional" and "another" and every other variation thereof should be construed as meaning "at least one additional."

[0018] As used herein, the term "plurality" should be understood as "two or more."

[0019] The term "configuration" or "setting" used to fulfill certain functions (and their corresponding variants) is to be understood in the sense of the present invention to mean that the corresponding device is already present in a configuration or setting in which it can implement the function or that the device can at least be set (i.e. configured) in such a way that it can implement the function after the corresponding setting. In this case, the configuration can be realized, for example, by correspondingly setting parameters of the process flow or setting switches or the like in order to activate or deactivate functionality or settings. In particular, the device can have a plurality of preset configurations or operating modes, so that the configuration can be carried out by selecting one of these configurations or operating modes.

[0020] Preferred embodiments of the method are described below, which can each be combined as desired with one another and with other aspects of the invention described further on, unless this is expressly excluded or technically impossible.

[0021] In some embodiments, the identified configuration of the at least one movable component includes the absolute position and orientation of the movable component in the region of the interior of the vehicle associated with the view with respect to a three-dimensional coordinate system. In other words, the movable component can be determined with respect to six degrees of freedom in the interior of the vehicle. This includes displacement along three axes (e.g., x, y, and z axes in an orthogonal coordinate system, such as a vehicle coordinate system) and rotation around these three axes. Thus, an accurate identification of the configuration of the movable component can be provided.

[0022] In some embodiments, the identified configuration of the at least one movable component comprises a relative position and orientation of the movable component with respect to at least one fixed part of the vehicle. When the configuration is identified with respect to a fixed part of the vehicle, an advantageous identification of the configuration of the movable component may be provided. For example, it may be determined how far the reclined seat back is from a (closed) door, interior trim or dashboard of the vehicle.

[0023] In some embodiments, the movable component includes at least one seat of the vehicle, wherein the identified configuration includes at least one of the seat position, seat rotation and inclination (or inclination angle) of the seat backrest. The seat of the vehicle is generally adjustable. This particularly relates to the driver's seat and the co-pilot seat. For example, the seat can be movable and adjustable in height. The seat backrest can also be adjusted. In the case of autonomous driving, it is also possible to rotate or turn the seat, especially the driver's seat and the co-pilot seat, so that they point in the direction of the center of the vehicle or in the direction opposite to the driving direction. Identifying such a configuration can be meaningful for certain vehicle functions, as explained in detail below. It can also be interesting to identify whether the backrest of the rear seat is folded (whether it is completely or partially folded). Like the front seat, in some vehicles, the rear seat can also be adjusted accordingly.

[0024] In some embodiments, the movable component comprises at least one accessory of the vehicle, wherein the identified configuration comprises the position and / or orientation of the accessory in the interior space of the vehicle. In addition to the seats, other components in the interior space may also be movable, which may also be relevant for the control of vehicle functions, for example when a movable accessory component may hinder a specific vehicle function. For example, it may be an accessory of a seat, such as an armrest of a seat or a movable footrest, a pivotable or reversible table top, a pivotable or reversible display or other movable accessories.

[0025] In some embodiments, the image acquisition device is calibrated by identifying the at least one fixed part. This is advantageous in particular when the image acquisition device is movable and does not have its own position sensor. The (automatic) calibration can then be performed simply based on the images captured by the image acquisition device, since the configuration of the fixed part of the vehicle is constant and known, for example, position, orientation, shape, etc.

[0026] In some embodiments, the position and / or orientation of the image acquisition device in the vehicle interior is determined by identifying the at least one fixed part. This can also be used for calibration. For example, the zero position of the image acquisition device can be determined in a simple manner in this way in order to simplify the identification of the configuration of the movable components in the vehicle interior.

[0027] In some embodiments, the method includes comparing the image data with (three-dimensional) construction data of the vehicle, in particular the interior space of the vehicle. This can be, for example, the CAD data of the vehicle. These data are known and can represent a three-dimensional model of the vehicle. By comparing this known model of the vehicle with the image data, the fixed parts and movable parts of the vehicle can be identified in a simple way, even if these fixed parts and movable parts are not completely included in the view, for example. For movable parts, it can be taken into account in particular that these movable parts are not movable arbitrarily, but only movable within a preset range. For example, a seat generally cannot be moved arbitrarily, but can only be moved on a certain path. The seat back can also generally only move within a specific angle range. All of this can be obtained from this construction data.

[0028] In some embodiments, at least one signal source is manipulated according to the identified configuration of the at least one movable component in order to cause the signal source to output a defined signal according to the manipulation. Vehicle functions can thereby be executed, activated or disabled. For example, it can be determined according to the configuration of the at least one movable component whether a specific airbag is deployed or disabled in a collision accident. For example, the driver's airbag can be disabled when the driver's seat is turned. It is also possible to disable a specific airbag, for example, when the display is flipped out so that the display enters the release area of ​​the airbag. Conversely, when the display is flipped in again, such an airbag can be activated. Another example can be the control of the vehicle ventilation, which is performed according to the configuration of the seat.

[0029] In some embodiments, the views each at least partially include a plurality of movable components of the interior of the vehicle, wherein the configuration of all movable components included in the views is recognized. Here, the movable components can be completely or only partially included in the views. In this way, a variety of changes in the configuration of the movable components in the interior can be fully responded to. Preferably, the views (each at least partially) include all movable components related to a specific vehicle function (e.g., the control of an airbag). Thus, the entire configuration of the vehicle interior can be easily recognized without the need for additional sensors.

[0030] A second aspect of the invention relates to a system for data processing, the system comprising at least one processor configured so that the processor implements the method according to the first aspect of the invention. The system also has at least one image acquisition device installed in the interior of a vehicle and configured to acquire a view of the interior of the vehicle.

[0031] In some embodiments, the image acquisition device is arranged in a movable part of the interior of the vehicle. This allows a flexible configuration of the interior. The image acquisition device does not have to be arranged in a fixed position. In particular, by means of the above-mentioned image evaluation, the configuration of the movable parts in the interior of the vehicle can be reliably identified even if the image acquisition device is movable.

[0032] In some embodiments, the image acquisition device is arranged in an interior rearview mirror of the vehicle. The arrangement in the interior rearview mirror is advantageous because it allows a good bird's-eye view of the interior (from the front and above). The visible area is in particular not obstructed, for example by the steering wheel or the vehicle occupants (if the image acquisition device is arranged, for example, in the dashboard). The image acquisition device can also be arranged in a concealed manner and does not interfere with the vehicle occupants. However, as mentioned above, the method can still be reliably implemented when the interior rearview mirror is moved.

[0033] In some embodiments, the image acquisition device is a 2D camera or a 3D camera. Two-dimensional or three-dimensional views are acquired accordingly. It can be an infrared camera or a time-of-flight camera (Time-of-Flight-Kamera, TOF camera) in particular. A corresponding infrared light source can be provided for the infrared camera (IR or NIR), so that the method can be reliably implemented even in the dark. When using a TOF camera, a three-dimensional view of the interior space can be acquired with the aid of a light propagation time method, in particular with the aid of a PMD sensor. Thus, for example, configuration changes of the interior space can be acquired in an accurate manner by determining the distance. It is understood that the image acquisition device can also be a camera that works with visible light (daylight), in particular in an RGB color space. The image acquisition device can be a wide-angle camera in particular, so as to obtain as large an area of ​​the interior space as possible as a view.

[0034] A third aspect of the invention relates to a computer program having instructions which, when executed on a system according to the second aspect, cause the system to carry out the method according to the first aspect.

[0035] The computer program can be stored in particular on a non-volatile data carrier. Preferably, this is a data carrier in the form of an optical data carrier or a flash memory module. This can be advantageous when the computer program itself is to be sold independently of the processor platform on which the one or more programs are to be executed. In another embodiment, the computer program can be present as a file on a data processing unit, in particular on a server, and can be downloaded via a data connection (for example, the Internet or a dedicated data connection such as a private network or a local area network). In addition, the computer program can have a plurality of separate program modules that act together.

[0036] The system according to the second aspect can accordingly have a program memory in which the computer program is stored. Alternatively, the system can also be configured to access an external computer program, for example a computer program available on one or more servers or other data processing units, via a communication connection, in particular to exchange data therewith, which are used during the execution of the method or the computer program or are outputs of the computer program.

[0037] The features and advantages described in relation to the first aspect of the present invention are also applicable to the other aspects of the present invention accordingly. This also applies to the features and advantages described in relation to the second aspect of the present invention.

[0038] Further advantages, features and possible applications of the present invention are apparent from the following detailed description in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In the figure:

[0040] Figure 1 a flow chart showing a method according to one embodiment; and

[0041] Figure 2 A view showing the interior of a vehicle. DETAILED DESCRIPTION

[0042] In the drawings, the same reference numerals are used throughout for identical or mutually corresponding elements of the present invention.

[0043] exist Figure 1 A method 100 for identifying a configuration of an interior of a vehicle is shown. Figure 2 The vehicle interior 2 is shown by way of example with four seats 11, 12, 13, 14. The method is executed in a data processing system of the vehicle 1, which may have a corresponding processor 5 (eg, ECU) and a memory 6 (in Figure 2 Schematically represented in ).

[0044] In step S1, a view of the interior 2 is first recorded with the camera 3 and corresponding image data are acquired for further processing by the system. The camera can be an infrared camera with a corresponding infrared light source, which can be operated in the dark. The camera 3 can also be a 3D camera, such as a TOF camera, which provides a three-dimensional view of the vehicle interior 2.

[0045] The camera 3 can be arranged in the interior rearview mirror 4 of the vehicle 1. Thus, the camera 3 can be arranged discreetly in the interior 2. Since the interior rearview mirror 4 is generally movable, the camera 3 must be calibrated. For this purpose, it is provided that the captured view of the interior 2 also captures fixed parts of the vehicle. This can be parts of the body or interior, such as the A-pillar 7, the B-pillar 8 or the center console 15.

[0046] The view also includes movable components of the vehicle interior 2, the configuration of which should be determined. These can be seats of the vehicle, such as the driver's seat 11, the front passenger seat 12 and the rear seats 13, 14. The position and / or orientation of the seats 11, 12, 13, 14 can be variable, including, for example, the tilting of the corresponding seat backrests. The movable components of the interior 2 can also be accessories in the interior 2, such as a reversible display 9 or a reversible table 10.

[0047] The image data are evaluated in step S2. In this case, fixed parts and movable components are identified in the view. For example, the position and orientation of the camera 3 in the vehicle coordinate system can be determined in a simple manner with the aid of CAD data from fixed parts such as the A-pillar 7, the B-pillar 8 or the center console 15. Then, by further image evaluation, for example by computer vision (i.e. machine vision) or by simple image recognition, movable parts, i.e. the configuration of seats 11, 12, 13, 14 or other parts such as the display 9 or the table 10, can be identified from the view (step S3).

[0048] Then, different vehicle functions can be controlled according to the detected interior space configuration (step S4). In particular, specific airbags can be activated or deactivated.

[0049] Although at least one exemplary embodiment has been described above, it should be noted that there are numerous variations of the exemplary embodiments described above. It should also be noted that the exemplary embodiments described are merely non-limiting examples and are not intended to limit the scope, applicability, or configuration of the apparatus and methods described herein. More specifically, the foregoing description provides a guide for those skilled in the art to implement at least one exemplary embodiment, and of course, various changes may be made to the modes of action and arrangements of the elements described in the exemplary embodiments without departing from the subject matter identified in the appended claims, respectively, and their legal equivalents. Reference numerals list

[0050] 100 Method for identifying the interior spatial configuration of a vehicle

[0051] 1 Vehicle

[0052] 2. Vehicle interior space

[0053] 3 cameras

[0054] 4Interior rearview mirror

[0055] 5 processors

[0056] 6 Memory

[0057] 7Fixed vehicle parts (A-pillar)

[0058] 8 Fixed vehicle parts (B-pillar)

[0059] 9. Flip-out display

[0060] 10. Reversible table top

[0061] 11 Driver's Seat

[0062] 12 passenger seats

[0063] 13 Left rear seat

[0064] 14Right rear seat

[0065] 15 Center console

Claims

1. A method (100) for identifying a configuration of an interior space (2) of a vehicle (1), wherein: The method comprises: - capturing image data by means of at least one image capture device (3), wherein the image data represent a view of the interior (2) of the vehicle (1), wherein the view at least partially includes at least one fixed part (7, 8, 15) of the vehicle (1) and at least one movable part (9, 10, 11, 12, 13, 14) of the interior (2) of the vehicle (1); - evaluating the image data, the evaluating comprising: - identifying the at least one fixed part (7, 8, 15) of the vehicle (1) in order to associate the view with a corresponding area of ​​the interior (2) of the vehicle (1); - Recognizing the arrangement of the at least one movable component (9, 10, 11, 12, 13, 14) of the interior (2) in a region of the interior (2) of the vehicle (1) associated with the view.

2. The method according to claim 1, wherein: The identified configuration of the at least one movable component (9, 10, 11, 12, 13, 14) comprises an absolute position and orientation of the movable component (9, 10, 11, 12, 13, 14) in relation to a three-dimensional coordinate system in a region of an interior (2) of a vehicle (1) associated with the view.

3. The method according to claim 1 or 2, wherein: The identified configuration of the at least one movable component (9, 10, 11, 12, 13, 14) comprises the relative position and orientation of the movable component (9, 10, 11, 12, 13, 14) with respect to at least one fixed part (7, 8, 15) of the vehicle (1).

4. A method according to any one of the preceding claims, wherein: The movable component comprises at least one seat (11, 12, 13, 14) of the vehicle (1), and the identified configuration comprises at least one of a seat position, a seat rotation and a backrest tilt of the seat.

5. A method according to any one of the preceding claims, wherein: The movable component comprises at least one accessory (9, 10) of the vehicle (1), and the identified configuration comprises the position and / or orientation of the accessory (9, 10) in the interior (2) of the vehicle (1).

6. A method according to any one of the preceding claims, wherein: The image acquisition device (3) is calibrated by identifying the at least one fixed part (7, 8, 15).

7. A method according to any one of the preceding claims, wherein: The position and / or orientation of an image acquisition device (3) in the interior (2) of a vehicle (1) is determined by detecting the at least one fixed part (7, 8, 15).

8. A method according to any one of the preceding claims, wherein: The method also includes comparing the image data with configuration data of the vehicle (2).

9. A method according to any one of the preceding claims, wherein: At least one signal source is actuated as a function of the detected configuration of the at least one movable component (9, 10, 11, 12, 13, 14) in order to cause the signal source to output a defined signal as a function of the actuation.

10. A method according to any one of the preceding claims, wherein: The views each at least partially include a plurality of movable components (9, 10, 11, 12, 13, 14) of an interior (2) of a vehicle (1), and the configuration of all movable components (9, 10, 11, 12, 13, 14) included in the views is identified.

11. A system for data processing, comprising at least one processor configured to implement the method according to any one of the preceding claims and at least one image acquisition device (3), the image acquisition device being installed in an interior space (2) of a vehicle (1) and being configured to acquire a view of the interior space (2) of the vehicle (1).

12. The system according to claim 11, wherein: The image acquisition device (3) is arranged in a movable part of the interior (2) of the vehicle (1).

13. The system according to claim 11 or 12, wherein: The image acquisition device (3) is arranged in an interior rearview mirror (4) of the vehicle (1).

14. The system according to any one of claims 10 to 12, wherein: The image acquisition device (3) is an infrared camera or a time-of-flight camera. 15 . A computer program comprising instructions which, when executed on a system according to claim 11 , cause the system to carry out the method according to claim 1 .

Images