Apparatus and method for controlling a child safety device of a vehicle

Abstract

An apparatus for controlling a child safety device of a vehicle (102) comprises at least one image generation system (128) of the vehicle (102). The camera (128) is designed to detect at least one image (200) containing a region depicting the interior of the vehicle and to generate image data corresponding to the image (200). The device further comprises a control unit (130) configured to process the image data, determine, based on the image data, whether an occupant intends to open a door (212) of the vehicle (102), determine, based on the image data, whether the occupant of the vehicle (102) is a child (206) or an adult, and determine, based on the image data, whether the occupant of the vehicle (102) is a child (206) or an adult. And blocking the opening of the door (212) if the control unit (130) has confirmed that the occupant intended to open the door (212) of the vehicle (102) is a child (206). Alternatively, the control unit is designed to allow the opening of the door (212) only if the control unit (130) has confirmed that the occupant intended to open the door (212) of the vehicle (102) is an adult.

Description

Technical Field

[0001] The present invention relates to a device for controlling a child safety device of a vehicle. The present invention also relates to a method for controlling a child safety device of a vehicle. Background Art

[0002] Child safety devices for the doors or windows of vehicles, especially cars, are known from the prior art. The child safety device prevents children, in particular, from opening the doors or windows of the vehicle, for example, during driving.

[0003] Document EP4027307A1 discloses a device for protecting children in a vehicle. The device is configured to classify an occupant as a child or an adult. The device is also configured to activate the child safety device of the vehicle door during driving or when another vehicle approaches the vehicle from behind when a child is recognized in the interior space of the vehicle.

[0004] Document US2010 / 138113A1 discloses another device for protecting children in a vehicle. The device is configured to recognize children in the interior space of the vehicle by means of a camera. The device is also configured to prevent the door opening device and the window opening device of the vehicle when a child is detected in the interior space of the vehicle.

[0005] Document CN108625709A discloses a system for operating a window lifter of a vehicle. The system is configured to recognize by means of a camera whether a child is operating the window lifter. The system is also configured to prevent the window from being fully opened when it is detected that a child is operating the window lifter.

[0006] Known child safety devices are often not user-friendly. Usually, the child safety device has to be manually activated and deactivated by the vehicle driver. When there is a child in the interior space of the vehicle, the inherently automatic child safety device completely prevents the opening of the doors and windows. This results in that even adult vehicle occupants cannot open some doors and windows from the inside when there is a child in the interior space of the vehicle. Summary of the Invention

[0007] Therefore, starting from the known prior art, the object of the present invention is to provide a device and a method for controlling a child safety device of a vehicle, which are more user-friendly than the hitherto known child safety devices.

[0008] This object is solved by a device having the features of claim 1 and by a method having the features of an independent method claim. Advantageous further developments are given in the dependent claims.

[0009] The proposed device for controlling a child safety device of a vehicle comprises at least one image generation system of the vehicle. The image generation system is configured to detect at least one image comprising a region depicting the interior space of the vehicle and to generate image data corresponding to the image. The device further comprises a control unit configured to process the image data, to determine based on the image data whether an occupant intends to open a door of the vehicle, to confirm based on the image data whether the occupant of the vehicle is a child or an adult, and to prevent the opening of the door if the control unit has confirmed that the occupant intending to open the door of the vehicle is a child. In an alternative embodiment, the control unit is configured to allow the opening of the door only if the control unit has confirmed that the occupant intending to open the door of the vehicle is an adult.

[0010] The proposed device identifies whether the occupant who wants to open the door of the vehicle is a child or an adult. For this purpose, the control unit first identifies the occupant in the image of the interior space of the vehicle detected by the image generation system. The control unit then confirms whether the occupant intends to open the door of the vehicle and whether the occupant is a child or an adult. If the device identifies that a child wants to open the door, the device prevents the door from opening. In the alternative embodiment, the device allows the door to open only if the device identifies that an adult wants to open the door. However, contrary to known child safety devices, the opening of the door is not permanently blocked, so that for example an adult occupant of the vehicle can open the door without having to switch off the child safety device beforehand. In particular, an adult occupant can open the door for a child without having to disable the child safety device beforehand. In other words, the device specifically prevents the door from opening only when a child tries to open the door. Thus, an adult occupant is not locked inside by the child safety device. This improves user-friendliness without compromising the safety of the child.

[0011] In the present context, the image generation system is understood to be any system configured to generate two-dimensional or three-dimensional images. In particular, the image generation system is understood to be a camera that detects two-dimensional images. Alternatively, the camera can be a time-of-flight camera that detects three-dimensional images. The camera particularly detects light in the visible spectrum and / or in the infrared spectrum. Alternatively, the image generation system can also consist of a radio radar or lidar system that generates a three-dimensional image of the interior space of the vehicle, particularly in the form of cloud points.

[0012] In another preferred embodiment, the image generation system is configured to detect a plurality of temporally consecutive images including a region depicting the interior space of the vehicle and to generate image data corresponding to the images. The control unit may be configured to confirm the movement of the occupant based on the image data and to determine whether the occupant intends to open the vehicle door based on the confirmed movement. In this embodiment, the image generation system detects an image or a video sequence. The movement performed by the occupant, for example with his hand, can be better recognized from the sequence. In order to be able to open the door, the occupant must perform a specific movement. For example, the occupant must grasp the operating mechanism for the door lock. Such a movement can be extracted from the sequence, for example, using a suitable machine learning method. Therefore, it is possible to confirm particularly reliably whether the occupant intends to open the vehicle door.

[0013] In another preferred embodiment, the position of at least one door opening device of the vehicle is stored in a storage area of the control unit. The control unit is configured to confirm the position of at least one arm of the occupant and the hand position of the hand of the arm based on the image data, and to determine that the occupant intends to open the vehicle door when the hand position coincides with the position of the door opening device. The confirmation of the arm position and the hand position can be performed, for example, using a suitable machine learning method. The door opening device is, for example, a door handle or a button for automatically opening the door. If the occupant moves his hand towards the door opening device, it can be assumed that the occupant intends to open the vehicle door. Therefore, in this embodiment, it is possible to confirm very reliably whether the occupant intends to open the vehicle door.

[0014] In another preferred embodiment, the control unit is configured to confirm the position of the arm and / or the hand position based on the image data and using a human-pose-estimation method. By means of the human-pose-estimation method, the position of the arm is reliably confirmed as a three-dimensional vector from the two-dimensional image of the image generation system. In particular, the hand position can also be confirmed with the help of the human-pose-estimation method, even if it is not visible in the image. Therefore, even when the position of the door opening device is not visible in the image, the control unit can reliably confirm in the current case whether the hand position coincides with the position of the door opening device.

[0015] In another preferred embodiment, the control unit is configured to confirm whether the vehicle is in motion and to prevent the door from being opened when the control unit confirms that the occupant is a child and the vehicle is in motion. It is particularly dangerous if a child opens the door during driving. Therefore, in this embodiment, the device is configured to prevent a child from opening the door, especially when the vehicle is in motion. Thereby, the device better protects the child in the interior space of the vehicle.

[0016] In another preferred embodiment, the device includes a sensor of the vehicle, which is configured to detect at least one image depicting an area outside the vehicle and generate sensor data corresponding to the image. The control unit is configured to process the sensor data and determine based on the sensor data whether another vehicle is approaching the vehicle, and to prevent the door from opening when the control unit has confirmed that the occupant is a child and another vehicle is approaching the vehicle. The sensor can be a camera. Alternatively, the sensor can also be a radio radar or lidar sensor. The sensor detects the environment of the vehicle. The control unit then confirms based on the sensor data detected by the sensor whether another vehicle is approaching the vehicle. In particular, if a child tries to leave the vehicle while another vehicle is approaching, a very dangerous situation may occur. For example, it may happen that the driver of another vehicle has to suddenly brake to avoid the child. In the worst case, the child may be seriously injured. Since in this embodiment the device prevents the door from being opened by a child when another vehicle is approaching, the child is better protected by the device.

[0017] In another preferred embodiment, the control unit is configured to confirm based on the image data and using a machine learning method whether the occupant is a child. In particular, the machine learning method is an age classification method, such as a face recognition method. Known age classification methods are able to reliably classify people as children and adults. Alternatively or additionally, the build of the occupant can also be determined by means of a human pose estimation method. Based on the build of the occupant, the occupant can then be classified as a child or an adult. To make the classification of the occupant more reliable, other features can be extracted from the image of the camera. For example, it can be determined by means of a suitable machine learning method whether the occupant is sitting in a child seat, etc., or the age of the occupant can be inferred from the style of the occupant's clothing. Using a machine learning method enables particularly reliable confirmation of whether the occupant who wants to open the door is a child.

[0018] In another preferred embodiment, the control unit is configured to determine based on the image data whether the occupant intends to operate the window lifter of the vehicle, and to prevent the window lifter from operating when the control unit has confirmed that the occupant is a child. In this embodiment, the device is additionally configured to prevent a child from operating the window lifter. This embodiment can in particular be further improved accordingly by all the features mentioned herein with regard to the opening of the door.

[0019] The present invention also relates to a method for controlling a child safety device of a vehicle. In this method, at least one image of a region including an area depicting the interior space of the vehicle is detected. Image data corresponding to the image is generated. Based on the image data, it is determined whether an occupant intends to open a door of the vehicle. Based on the image data, it is confirmed whether the occupant of the vehicle is a child or an adult. If it is confirmed that the occupant intending to open the door of the vehicle is a child, the opening of the door is blocked.

[0020] This method has the same advantages as the claimed device. In particular, this method can be further improved by using the features of the dependent claims for the device. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Embodiments of the present invention will be described in detail below with reference to the accompanying drawings. In the drawings:

[0022] Figure 1 A perspective schematic view showing the cockpit of a vehicle;

[0023] Figure 2 A schematic view showing an image of the interior space of a vehicle taken by a camera; and

[0024] Figure 3 A flowchart showing a method for controlling a child safety device of a vehicle. DETAILED DESCRIPTION

[0025] Figure 1 A perspective schematic view showing the cockpit 100 of a vehicle 102.

[0026] A vehicle driver 104 is sitting on a driver's seat 106 of the vehicle 102. The cockpit 100 includes a central information display 108 (CID), a head-up display 110, and a graphic combination meter 112, which are arranged on the instrument panel 114 of the vehicle 102. The cockpit 100 also includes a steering wheel 116 having operating elements 118, a gear selection switch 120, pedals 122, and an input unit 124 having a rotary wheel and button functions and / or a touch input area. The input unit 124 is also referred to as an ErgoCommander.

[0027] In an upper region of the front windshield 126 of the vehicle 102, a camera 128 as an image generation system is arranged. The camera 128 is arranged and oriented such that the camera 128 can detect an image of an area including an area depicting the interior space of the vehicle 102. In the following, an exemplary image 200 of the camera 128 will be described. The camera 128 is particularly configured to detect a plurality of temporally consecutive images, generate image data corresponding to the images, and transmit it to the control unit 130. Figure 2

[0028] ​The control unit 130 has a data output 132 and a data input 134, which are used to connect to other units of the vehicle 102, such as other sensors, input and output units, and control units of auxiliary systems. In particular, the control unit 130 is connected to a sensor 136, which is configured to detect at least one image depicting an area outside the vehicle 102, generate sensor data corresponding to the image, and transmit the sensor data to the control unit 130. The sensor 136 is in particular a radar or lidar sensor or another camera. Based on the sensor data, the control unit 130 can in particular confirm whether another vehicle is approaching the vehicle 102.

[0029] The control unit 130 is configured to implement a method for controlling a child safety device of the vehicle 102. In this method, it is determined by means of the camera 128 whether a child or an adult intends to open the door of the vehicle 102 or operate the window lifter of the vehicle 102. When it is confirmed that a child intends to open the door or operate the window lifter, the control unit 130 prevents the door or window from being opened. Alternatively, the control unit 130 is configured to allow the door to be opened only when an adult opens the door. The method is described in detail below with the help of Figure 2 and Figure 3 This method is described in detail.

[0030] Figure 2 FIG. 200 shows a schematic view of an image of the interior space of the vehicle 102 taken by the camera 128.

[0031] The camera 128 has a field of view of 120 degrees. In other embodiments, the camera 128 may also have a field of view in the range of 150 degrees or more. The camera 128 is directed towards the driver's seat 106 of the co-driver seat 202 and the rear bench 204 of the vehicle 102. In Figure 2 the case shown, a child 206 is sitting on the rear bench 204 of the vehicle 102. In addition to the child 206, the vehicle driver 104 is still located in the interior space of the vehicle and is sitting on the driver's seat 106 of the vehicle 102.

[0032] Figure 2 The detail view in FIG. shows the area 208 around the door handle 210 of the door 212 in the rear area of the vehicle 102 on the side where the child 206 is sitting. This area includes, in addition to the door handle 210 forming the door opening device, an operating element 214 of the window lifter for the window of the door 212. The area 208 around the door handle 210 is blocked by the co-driver seat 202 of the vehicle 102 in the image of the camera 128. The positions of the door handle 210 and the operating element 214 of the window lifter are stored in the storage area of the control unit 130.

[0033] To determine whether one of the occupants intends to open door 212, control unit 130 first determines the hand position of the occupant from the image data. To be able to determine the hand position in coordinates fixed to the vehicle, control unit 130 is in particular configured to identify a fixed reference point, such as a door handle button or switch, in the interior space of the vehicle based on the image data. The reference point identification can in particular be carried out using CAD data of the interior space of the vehicle.

[0034] To determine the hand position, control unit 130 in particular implements a human pose estimation method. The human pose estimation method has the advantage that the hand position of the occupant can be determined even when the hand is not visible in the image 200 of camera 128, as in the case shown in Figure 2 for example, the hand is blocked by the front passenger seat 202 or the driver's seat 106. When the hand position of the occupant coincides with one of the positions of the door opening device stored in the storage area of control unit 130, control unit 130 determines that the occupant is attempting to open door 212.

[0035] Control unit 130 then determines whether the occupant who is about to open door 212 is a child 206 or an adult. For this purpose, control unit 130 implements a face recognition method or another suitable machine learning method using the image data. If control unit 130 determines that child 206 is attempting to open door 212, control unit 130 prevents door 212 from being opened. In particular, if vehicle 102 is in motion, or control unit 130 determines based on the sensor data of sensor 136 that another vehicle 102 is approaching vehicle 102, control unit 130 prevents door 212 from being opened. Alternatively, control unit 130 only allows door 212 to be opened when an adult opens it.

[0036] In a similar manner, control unit 130 can also prevent child 206 from operating the window regulator of vehicle 102 to open window 216. For this purpose, control unit 130 determines whether the identified hand position of child 206 coincides with the position of the operating element 214 of the window regulator stored in the storage area of control unit 130. If the hand position of child 206 coincides with the position of window regulator control 214, control unit 130 determines that child 206 is attempting to operate the window regulator. Control unit 130 can thus in particular prevent window 216 from being opened by means of the window regulator.

[0037] Figure 3 A flowchart showing a method for controlling a child safety device of vehicle 102.

[0038] The method starts at step S300. At step S302, the control unit 130 controls the camera 128 such that the camera 128 detects at least one image 200 depicting the interior space of the vehicle. Image data corresponding to the image 200 is generated by the camera 128 and transmitted to the control unit 130. At step S304, the control unit 130 determines based on the image data whether one of the occupants of the vehicle 102 is intending to open one of the doors of the vehicle 102 or operate one of the window regulators. At step S306, the control unit 130 then confirms based on the image data whether the occupant intending to open one of the doors of the vehicle 102 or operate one of the window regulators is a child or an adult. At step S308, if the control unit 130 determines that the occupant is a child, the control unit 130 locks the opening of the door or window to protect the child. Alternatively, at step S308, the control unit only allows the opening of the door or window if the control unit 130 determines that an adult wants to open the door or window. The method ends at step S310.

[0039] In the embodiment according to Figures 1 to 3 the description, the camera 128 and the control unit 130 form a device for controlling the child safety device of the vehicle 102. Other elements and features shown in Figures 1 to 3 and mentioned in the foregoing description may be part of the device.

[0040] List of reference numerals

[0041] 100 Cockpit

[0042] 102 Vehicle

[0043] 104 Vehicle driver

[0044] 106 Driver's seat

[0045] 108, 110, 112 Display elements

[0046] 114 Instrument panel

[0047] 116 Steering wheel

[0048] 118 Operating element

[0049] 120 Gear selection switch

[0050] 122 Pedal

[0051] 124 Input unit

[0052] 126 Windshield

[0053] 128 Camera

[0054] 130 Control unit

[0055] 132 Data output terminal

[0056] 134 Data input terminal

[0057] 136 Sensor

[0058] 200 Image

[0059] 202 Co-pilot seat

[0060] 204 Rear bench seat

[0061] 206 Child

[0062] 208 Area

[0063] 210 Door handle

[0064] 212 Door

[0065] 214 Operating element

Claims

1. A device for controlling a child safety device of a vehicle (102), the device comprising: at least one image generation system (128) of the vehicle (102), the image generation system being configured to detect at least one image (200) including a region depicting the interior space of the vehicle and to generate image data corresponding to the image (200), a control unit (130) configured to process the image data, determine based on the image data whether an occupant intends to open a door (212) of the vehicle (102), confirm based on the image data whether the occupant of the vehicle (102) is a child (206) or an adult, and if the control unit (130) has confirmed that the occupant intending to open the door (212) of the vehicle (102) is a child (206), prevent the opening of the door (212), or allow the opening of the door (212) only when the control unit (130) has confirmed that the occupant intending to open the door (212) of the vehicle (102) is an adult.

2. The device according to claim 1, wherein, the image generation system (128) is configured to detect a plurality of temporally consecutive images including a region depicting the interior space of the vehicle and to generate image data corresponding to the images; and the control unit (130) is configured to confirm the movement of the occupant based on the image data and determine based on the confirmed movement whether the occupant intends to open the door (212) of the vehicle (102).

3. The device according to claim 1 or 2, wherein, the position of at least one door opening device (210) of the vehicle (102) is stored in a storage area of the control unit (130); and the control unit (130) is configured to confirm the position of at least one arm of the occupant and the hand position of the hand of the arm based on the image data, and is configured to determine that the occupant intends to open the door (212) of the vehicle (102) when the hand position coincides with the position of the door opening device (210).

4. The device according to claim 3, wherein, the control unit (130) is configured to confirm the position of the arm and / or the hand position based on the image data and in the case of using a human pose estimation method.

5. The device according to any one of the preceding claims, wherein, the control unit (130) is configured to: confirm whether the vehicle (102) is in motion, and when the control unit (130) has confirmed that the occupant is a child (206) and the vehicle (102) is in motion, prevent the opening of the door (212).

6. The device according to any one of the preceding claims, the device including a sensor (136) of the vehicle (102), the sensor being configured to detect at least one image including a region depicting the exterior of the vehicle (102) and to generate sensor data corresponding to the image; wherein, The control unit (130) is configured to process the sensor data and determine, based on the sensor data, whether another vehicle (102) is approaching the vehicle (102), and to prevent the door (212) from being opened when the control unit (130) has confirmed that the occupant is a child (206) and another vehicle (102) is approaching the vehicle (102).

7. The device according to any one of the preceding claims, wherein, the control unit (130) is configured to confirm, based on the image data and using a machine learning method, whether the occupant is a child (206).

8. The device according to any one of the preceding claims, wherein, the control unit (130) is configured to determine, based on the image data, whether the occupant intends to operate the window regulator of the vehicle (102), and to lock the window regulator when the control unit (130) has confirmed that the occupant is a child (206).

9. A method for a child safety device for controlling a vehicle (102), in which at least one image comprising a region depicting the interior space of the vehicle is detected, image data corresponding to the image is generated, it is determined, based on the image data, whether an occupant intends to open the door (212) of the vehicle (102), it is confirmed, based on the image data, whether the occupant of the vehicle (102) is a child (206) or an adult, and if it is confirmed that the occupant intending to open the door (212) of the vehicle (102) is a child (206), the opening of the door (212) is prevented.

Images