Methods for monitoring infants in vehicles and systems for monitoring infants in vehicles

Abstract

This provides a method that allows caregivers to monitor the condition of infants in the back seat of a vehicle in real time, even while driving. [Solution] The method includes the steps of: a rear-seat infant monitoring device taking a picture toward a child seat in the rear seat of the vehicle and acquiring video and audio signals; inputting the video and audio signals into an artificial intelligence engine, analyzing the video, detecting the facial expressions of the infant, analyzing the emotions of the infant based on the audio characteristics, and generating a notification event corresponding to the current emotions of the infant; the rear-seat infant monitoring device transmitting the notification event to a front-seat display device; the front-seat display device acquiring video and audio signals by streaming them from a video and audio processing module based on the notification event and displaying them; and the front-seat display device requesting the rear-seat infant monitoring device to stream video and audio packets based on the notification event, and playing back the video and audio signals of the infant on the front-seat display device.

Description

Technical Field

[0001] The present invention relates to a system and method for monitoring passengers in a vehicle, and particularly to a system and method for monitoring infants in a vehicle.

Background Art

[0002] Infants are small and delicate and cannot take care of themselves, so they always need to be accompanied and taken care of by a caregiver. Usually, the activity time of infants is not constant. Sometimes the caregiver plays with the infant, and sometimes the infant plays alone while the caregiver attends to their own business. In such a situation where mobility is required, even a moment of inattention by the caregiver can lead to unfortunate accidents. Therefore, great care must always be taken when taking care of infants.

[0003] As a conventional method, a caregiver installs a camera near the infant's bed or activity space to photograph the infant's actions. On the other hand, monitoring software is installed on the user's mobile terminal and communicatively connected to the camera to determine the motion state of the infant in the video using a video recognition function. When an abnormality is detected, the mobile terminal immediately emits a warning sound so that the caregiver can interrupt the work at hand and check on the infant.

[0004] When a caregiver drives a vehicle with an infant on board, regulations require that a child seat for infants be installed in the rear seat to reduce the risk in case of a traffic accident and ensure the safety of the infant. However, when a caregiver drives alone with an infant, it is dangerous to directly turn around and check on the infant in the rear seat while driving in the front seat, which poses a problem for safe driving. Also, it is difficult to check the condition of the infant in the rear seat while the caregiver is concentrating on driving. In such a situation, even if an unexpected accident occurs to the infant in the rear seat, such as the seat belt getting caught around the neck and the infant being unable to make a sound, the caregiver may not be able to immediately detect the abnormality and take emergency measures.

[0005] Therefore, enabling caregivers to monitor the condition of infants in the back seat in real time, even while driving a vehicle, is an extremely important issue. [Overview of the project] [Effects of the Invention]

[0006] One embodiment of the present invention is a method for monitoring infants in a vehicle, which is applied to a system for monitoring infants in a vehicle, and the system for monitoring infants in a vehicle includes a rear-seat infant monitoring device and a front-seat display device connected to the rear-seat infant monitoring device. The method for monitoring infants in a vehicle includes the following steps: (a) The video and audio acquisition module of the rear-seat infant monitoring device takes a picture toward the child seat in the rear seat of the vehicle and acquires a video and audio signal. (b) The video and audio processing module of the rear-seat infant monitoring device inputs the video and audio signal to an artificial intelligence engine, which analyzes the video of the video and audio signal, detects the facial expression of the infant, analyzes the emotional state of the infant based on the audio characteristics, and generates a notification event corresponding to the current emotion of the infant. (c) The rear-seat processing module of the rear-seat infant monitoring device receives the notification event and transmits it to the front-seat display device. (d) The video and audio playback module of the front-seat display device streams and acquires the video and audio signal from the video and audio processing module based on the notification event and displays it. (e) The front seat processing module of the front seat display device instructs the video and audio playback module to stream and play based on a notification event, thereby playing the video and audio signals of the infant on the video and audio playback module located in the front seat of the vehicle.

[0007] One embodiment of the present invention is a system for monitoring infants in a vehicle, and includes a rear-seat infant monitoring device and a front-seat display device connected to the rear-seat infant monitoring device. The rear-seat infant monitoring device includes a video / audio acquisition module, a video / audio processing module, and a rear-seat processing module. The video / audio acquisition module takes images toward the child seat in the rear of the vehicle and acquires video and audio signals. The video / audio processing module includes an artificial intelligence engine, which inputs the video and audio signals into the artificial intelligence engine, analyzes the infant's facial expressions, and analyzes the infant's emotional state based on audio characteristics, thereby generating notification events corresponding to the infant's current emotions. The rear-seat processing module is connected to the video / audio processing module and receives notification events. The front-seat display device includes a video / audio playback module and a front-seat processing module. The video / audio playback module streams and acquires video and audio signals from the video / audio processing module and displays them. The front seat processing module is connected to the video and audio playback module and, based on notification events, instructs the video and audio playback module to stream and play, thereby playing the video and audio signals of the infant on the video and audio playback module located in the front seat of the vehicle. [Brief explanation of the drawing]

[0008] [Figure 1] This is a block diagram of a system for monitoring infants inside a vehicle. [Figure 2] This is a flowchart showing how to monitor infants and toddlers inside a vehicle. [Examples]

[0009] The present invention will be described in more detail below with reference to the drawings and examples, so that those skilled in the art may better understand and implement the present invention. However, the examples given are not intended to limit the present invention.

[0010] Typically, the caregiver sits in the front seat of the vehicle while driving, and the infant is placed in a child seat in the back seat. Therefore, it is difficult for the caregiver or user to constantly pay attention to the infant's condition in the back seat. This invention provides a solution to this situation.

[0011] Please refer to Figure 1, which is a block diagram of a system for monitoring infants inside a vehicle.

[0012] The system for monitoring infants inside the vehicle includes a rear-seat infant monitoring device 1 and a front-seat display device 2.

[0013] The rear-seat infant monitoring device 1 is installed on the roof of the vehicle and positioned to capture and acquire sensing information of the infant's face, voice, and the in-vehicle environment while the infant is sitting in the child seat. For example, the rear-seat infant monitoring device 1 may be integrated with the rear roof lamp or fixed to a bracket provided near it, ensuring stable mounting of the rear-seat infant monitoring device 1 and preventing camera angle deviations. Alternatively, a sliding rail may be provided on the roof inside the vehicle to allow the position of the rear-seat infant monitoring device 1 to be freely adjusted.

[0014] The rear-seat infant monitoring device 1 may be installed near the power supply unit in the rear seat, thereby stabilizing the power supply to the rear-seat infant monitoring device 1 and preventing abnormal power outages.

[0015] The front seat display device 2 is installed in the front seat of the vehicle and is configured to be easily viewed by the caregiver or the passenger. The front seat display device 2 may be a separate display, a mobile device (e.g., a smartphone or tablet), or an in-vehicle infotainment system (IVI system).

[0016] The rear-seat infant monitoring device 1 is connected to the front-seat display device 2. The rear-seat infant monitoring device 1 transmits detected events of infants in the rear seats to the front-seat display device 2 and notifies the caregiver of the infant's current condition. Further details are described below.

[0017] The rear-seat infant monitoring device 1 includes a video / audio acquisition module 11, a video / audio processing module 12, a rear-seat processing module 13, a rear-seat communication module 14, a temperature sensor 15, and an air quality sensor 16.

[0018] The video and audio acquisition module 11 is mounted on the roof of the vehicle and takes images directed towards the child seat in the rear seat of the vehicle to acquire video and audio signals of the infant. The video and audio acquisition module 11 may be a camera. Alternatively, the video and audio acquisition module 11 may be a visible light and / or infrared sensor module.

[0019] The video / audio processing module 12 is connected to the video / audio acquisition module 11. The video / audio processing module 12 includes an artificial intelligence engine 121, an encoder 123, and a streaming server 125.

[0020] The video / audio processing module 12 inputs the video signal of the video / audio signal to the artificial intelligence engine 121, which then analyzes the infant's facial expressions. Meanwhile, the video / audio processing module 12 inputs the audio signal of the video / audio signal to the artificial intelligence engine 121, which then analyzes the infant's emotional state based on the audio characteristics. Through the analysis function of the artificial intelligence engine 121, the video / audio processing module 12 generates notification events corresponding to the infant's current emotions.

[0021] For example, the artificial intelligence engine 121 analyzes video to detect the infant's face and continuously tracks its movements. If face tracking is interrupted, for example, if the infant disappears from the video or is obscured by an unknown object, the artificial intelligence engine 121 generates a notification event. Specifically, this could occur if the infant slips out of the car seat or if the seat belt gets tangled around its neck. Furthermore, the artificial intelligence engine 121 analyzes video to detect the infant's facial expressions and can identify the infant's current emotional state from expressions such as sleeping, frowning, wide-eyed and curious, pursed lips, and open-mouthed and smiling. Caregivers can pre-set the system to generate a notification event when the infant exhibits a specific emotion, allowing the caregiver to view or save the video and audio. The artificial intelligence engine 121 can also analyze video to detect abnormal behavior in the infant, such as "not moving for a long time," "turning the body around," or "twisting unnaturally." If detected, the artificial intelligence engine 121 generates a notification event to inform the caregiver of the current status of the infant in the back seat.

[0022] The artificial intelligence engine 121 may be an artificial intelligence model trained using various neural networks, such as artificial neural networks (ANN), deep neural networks (DNN), convolutional neural networks (CNN), and recurrent neural networks (RNN), and can perform tasks such as recognizing the faces and expressions of infants, recognizing upper body body language, and recognizing voice emotions.

[0023] The video and audio processing module 12 may be a graphics processor, and perform video optimization processing such as automatic exposure, automatic white balance, high dynamic range adjustment, wide dynamic range adjustment, etc. on the video input from the video and audio acquisition module 11. When the vehicle is running, the illuminance inside the vehicle varies due to changes in the external environment, which affects the shooting of the in-vehicle video. Therefore, the video and audio processing module 12 performs optimization processing on the video, corrects the difference in brightness between each video, thereby improving the accuracy of video recognition and analysis by the artificial intelligence engine 121.

[0024] The rear seat infant monitoring device 1 can optionally include an infrared supplementary light module (not shown). When the video and audio processing module 12 detects insufficient illuminance, it activates the infrared supplementary light module to irradiate infrared light to compensate for the insufficient illuminance and improve the quality of the video captured by the video and audio acquisition module 11.

[0025] Furthermore, when the vehicle is running, since it is in a space with a lot of environmental noise, the artificial intelligence engine 121 uses an artificial intelligence-based noise reduction program to remove the noise of external vehicles and the environment. The voice signal after the noise reduction process helps to improve the accuracy of the analysis of the emotional state of the infant based on the voice characteristics by the artificial intelligence engine 121.

[0026] On the other hand, the artificial intelligence engine 121 analyzes the voice signal to identify the utterance of the infant. As an example, the artificial intelligence engine 121 uses an infant voice translation model (a pre-trained artificial intelligence model) to identify what emotion the voice uttered by the infant means or what it is trying to convey, and provides it as reference information to the caregiver.

[0027] Meanwhile, the encoder 123 of the video / audio processing module 12 receives the video / audio signals acquired by the video / audio acquisition module 11 via the artificial intelligence engine 121, and encodes and compresses the video / audio signals to generate data in streaming packet format (e.g., HLS (HTTP Live Streaming), MPEG-DASH (Dynamic Adaptive Streaming over HTTP), RTMP (Real-Time Messaging Protocol), and RTSP (Real Time Streaming Protocol)).

[0028] The streaming server 125 of the video / audio processing module 12 causes the rear-seat infant monitoring device 1 to function as a streaming service server and provides video and audio signals to the front-seat display device 2 through the streaming service.

[0029] The rear seat processing module 13 can be configured as a System on a Chip (SoC). The rear seat processing module 13 is connected to the video / audio processing module 12 and the rear seat communication module 14. The rear seat processing module 13 is used to initialize and configure the rear seat infant monitoring device 1 and to determine and process notification events. When it receives a notification event from the video / audio processing module 12, it transmits a notification or alarm to the front seat display device 2 via the rear seat communication module 14.

[0030] The rear seat processing module 13 can also detect whether an infant or young child has been left inside the vehicle. For example, if the rear seat processing module 13 detects that the vehicle's engine has stopped, but the video and / or audio processing module 12 continues to detect the face and / or voice of an infant or young child, and the rear seat processing module 13 continues to receive notification events, the rear seat processing module 13 will emit a warning signal, and the video and audio playback module 21 of the front seat display device 2 will issue several types of warnings, such as a warning sound, flashing text on the lock screen, or flashing screen backlight. The caregiver can only end these warnings by pressing a confirmation button.

[0031] The temperature sensor 15 is connected to the rear seat processing module 13 and is used to measure the temperature inside the vehicle. If the rear seat processing module 13 determines that the temperature inside the vehicle exceeds a comfortable temperature range (e.g., 22 to 24 degrees Celsius), it sends a notification event to the front seat display device 2, prompting the caregiver to adjust the temperature of the vehicle's air conditioner, thereby preventing heatstroke or colds in infants and young children.

[0032] The air quality sensor 16 is connected to the rear seat processing module 13 and is used to detect the quality of the air inside the vehicle. The rear seat processing module 13 detects when the quality of the air inside the vehicle exceeds the comfortable range (e.g., carbon dioxide concentration exceeds 1000 ppm, or PM2.5 concentration exceeds 12 μg / m³). 3 If it is determined that the threshold has been exceeded, a notification event is sent to the front seat display device 2, prompting the caregiver to adjust the ventilation intensity of the in-car air conditioner so that the infant can breathe fresh air.

[0033] The front seat display device 2 includes a video / audio playback module 21, a front seat processing module 22, a storage module 23, and a front seat communication module 24.

[0034] The front seat communication module 24 of the front seat display device 2 is connected to the rear seat communication module 14 of the rear seat infant monitoring device 1. The front seat communication module 24 and the rear seat communication module 14 may be connected by any wireless communication method, such as Wi-Fi or Bluetooth, or by any wired communication method, such as Ethernet or USB.

[0035] The video and audio playback module 21 is used to reproduce video and audio signals transmitted from the rear-seat infant monitoring device 1. The video and audio playback module 21 includes a streaming client 211, a decoder 213, and a display module 215.

[0036] The streaming client 211 causes the front seat display device 2 to function as a client for the streaming service and receives video and audio signals through the streaming service of the rear seat infant monitoring device 1.

[0037] The decoder 213 decodes the received video and audio signals, and outputs the decoded video and audio signals to the display module 215 for playback.

[0038] The front seat processing module 22 is connected to the video / audio playback module 21. The front seat processing module 22 receives notification events from the rear seat infant monitoring device 1 via the front seat communication module 24, and displays a warning message on the display module 215 or emits a warning sound on the audio playback module (not shown). The front seat processing module 22 can be configured as a system on a chip (SoC).

[0039] The memory module 23 is connected to the front seat processing module 22 and the front seat communication module 24 and is used to store video and audio signals from the rear seat infant monitoring device 1. The memory module 23 stores the video and audio signals corresponding to each notification event, creating a record for each notification event. For example, if the infant's facial expression is smiling, after the rear seat infant monitoring device 1 generates a notification event, the video and audio signals are streamed to the front seat display device 2, played back by the video and audio playback module 21, and simultaneously stored in the memory module 23.

[0040] Figure 2 is a flowchart of a method for monitoring infants in a vehicle. This monitoring can be performed using the system for monitoring infants in a vehicle shown in Figure 1.

[0041] In step S31, the video and audio acquisition module 11 takes images toward the child seat in the rear seat of the vehicle and acquires video and audio signals.

[0042] In step S32, the video and audio processing module 12 inputs video and audio signals to the artificial intelligence engine 121.

[0043] In step S33, the artificial intelligence engine 121 analyzes the infant's facial expressions and emotional state based on voice characteristics, and generates a notification event corresponding to the infant's current emotions.

[0044] In step S34, the rear seat processing module 13 receives a notification event.

[0045] In step S35, the streaming client 211 of the video / audio playback module 21 receives video and audio signals via streaming from the streaming server 125 of the video / audio processing module 12, and displays them.

[0046] In step S36, the front seat processing module 22 instructs the video / audio playback module 21 to stream and play based on the notification event, thereby playing the video and audio signals of the infant on the video / audio playback module 21 located in the front seat of the vehicle.

[0047] As described above, the system and method for monitoring infants in a vehicle according to the present invention are applied to the in-vehicle environment and monitor the condition of infants sitting in child seats in the rear seats, and notify the driver of the infant's condition in real time. Whether the infant is in an abnormal state (requiring the caregiver to pay close attention) or in a normal state (allowing the caregiver to share in the infant's joyful feelings and enjoy the infant's adorable appearance), the caregiver can fully understand the infant's condition in the rear seat without having to look back. This allows the caregiver to balance public safety while driving with humane considerations and the protection of the infant's life, thereby enhancing a double layer of safety.

[0048] The above are merely specific embodiments of the present invention and are not intended to limit the scope of the claims of the present invention. Accordingly, any equivalent modifications made by applying the content of the present invention shall be included within the scope of protection of the present invention. [Explanation of Symbols]

[0049] 1: Rear-seat infant monitoring device 11: Video and audio acquisition module 12: Video and audio processing module 121: Artificial Intelligence Engine 123: Encoder 125: Streaming Server 13: Rear seat processing module 14: Rear seat communication module 15: Temperature sensor 16: Air quality sensor 2: Front seat display device 21: Video and audio playback module 211: Streaming Client 213: Decoder 215: Display Module 22: Front seat processing module 23: Memory Module 24: Front seat communication module S31~S36: Step

Claims

1. This system is applicable to a system for monitoring infants in a vehicle, including a rear-seat infant monitoring device and a front-seat display device connected to the rear-seat infant monitoring device. The video and audio acquisition module of the rear-seat infant monitoring device takes a picture of the child seat in the rear seat of the vehicle and acquires video and audio signals (a), (b) The video and audio processing module of the rear-seat infant monitoring device inputs the video and audio signals to an artificial intelligence engine, the artificial intelligence engine analyzes the video of the video and audio signals, detects the facial expressions of the infant, analyzes the emotional state of the infant based on the audio characteristics, and generates a notification event corresponding to the infant's current emotions. (c) the rear seat processing module of the rear seat infant monitoring device receives the notification event and transmits it to the front seat display device, The video and audio playback module of the front seat display device acquires the video and audio signals from the video and audio processing module via streaming based on the notification event, and displays them (d), A method for monitoring an infant in a vehicle, comprising the step (e) in which the front seat processing module of the front seat display device instructs the video and audio playback module to stream and play based on the notification event, thereby playing the video and audio signals of the infant on the video and audio playback module located in the front seat of the vehicle.

2. Step (b) is, The artificial intelligence engine detects the infant's face and continuously tracks the infant's facial movements (b11), A method for monitoring an infant in a vehicle according to claim 1, comprising the step (b12) that the artificial intelligence engine generates the notification event when the infant's face disappears from the video or is obscured by an unknown object.

3. A method for monitoring an infant in a vehicle according to claim 1, wherein the notification events include "the infant slips out of the child seat," "the seat belt gets tangled around the infant's neck," "the infant is sleeping," "the infant is frowning," "the infant is looking around with wide-open eyes and interest," "the infant is pursing their lips," "the infant is smiling with their mouth open," "the infant remains motionless for a long period of time," "the infant is turning their body over," or "the infant is twisting their body in an unnatural way."

4. Step (c) is Step (c1) when the notification event received by the rear seat processing module indicates that an infant is left inside the vehicle, the rear seat processing module transmits a warning signal to the front seat display device, A method for monitoring an infant in a vehicle according to claim 1, comprising the step (c2) of the video and audio playback module of the front seat display device providing a plurality of types of warnings, including at least one of a warning sound, flashing characters on the lock screen, and flashing the screen backlight.

5. The steps include: the rear-seat infant monitoring device's temperature sensor measures the temperature inside the vehicle, and the rear-seat processing module determines that the temperature inside the vehicle exceeds a comfortable temperature range, and transmits the notification event to the front-seat display device; A method for monitoring an infant in a vehicle according to claim 1, further comprising the steps of: the rear-seat infant monitoring device's air quality sensor detects the quality of the air inside the vehicle; and the rear-seat processing module determines that the quality of the air inside the vehicle exceeds a comfortable range, and transmits the notification event to the front-seat display device.

6. A video and audio acquisition module that captures video and audio signals by shooting towards the child seat in the rear of the car, A video and audio processing module includes an artificial intelligence engine, inputs the video and audio signals to the artificial intelligence engine, analyzes the facial expressions of the infant, analyzes the emotional state of the infant based on the audio characteristics, and generates a notification event corresponding to the infant's current emotions. A rear-seat infant monitoring device including a rear-seat processing module connected to the aforementioned video and audio processing module and receiving the aforementioned notification event, A video and audio playback module that streams and acquires the video and audio signals from the aforementioned video and audio processing module and displays them, The system includes a front seat processing module connected to the aforementioned video and audio playback module, which, based on the notification event, instructs the video and audio playback module to stream and play, thereby playing the video and audio signals of the infant on the video and audio playback module located in the front seat of the vehicle, A system for monitoring infants in a vehicle, further comprising a front-seat display device connected to the rear-seat infant monitoring device.

7. The system for monitoring an infant in a vehicle according to claim 6, wherein the artificial intelligence engine detects the infant's face, continuously tracks the movement of the infant's face, and generates the notification event when the infant's face disappears from the image or is obscured by an unknown object.

8. A system for monitoring an infant in a vehicle according to claim 6, wherein the notification events include "the infant slips out of the child seat," "the seat belt gets tangled around the infant's neck," "the infant is sleeping," "the infant is frowning," "the infant is looking around with wide-open eyes and interest," "the infant is pursing their lips," "the infant is smiling with their mouth open," "the infant remains motionless for a long period of time," "the infant is turning their body upside down," or "the infant is twisting their body in an unnatural way."

9. The system for monitoring an infant in a vehicle according to claim 6, wherein, when the notification event received by the rear seat processing module indicates that an infant has been left in the vehicle, the rear seat processing module transmits a warning signal to the front seat display device, and the video and audio playback module of the front seat display device provides a plurality of warnings, including at least one of a warning sound, flashing characters on the lock screen, and flashing of the screen backlight.

10. The aforementioned rear-seat infant monitoring device, A temperature sensor connected to the rear seat processing module and used to measure the temperature inside the vehicle, The system further includes an air quality sensor connected to the rear seat processing module and for detecting the quality of the air inside the vehicle, The system for monitoring infants in a vehicle according to claim 6, wherein the rear seat processing module transmits the notification event to the front seat display device when it determines that the temperature inside the vehicle has exceeded a comfortable temperature range, and the rear seat processing module transmits the notification event to the front seat display device when it determines that the quality of the air inside the vehicle has exceeded a comfortable temperature range.

Images