In-vehicle infant-and-toddler monitoring method and in-vehicle infant-and-toddler monitoring system

Abstract

An in-vehicle infant-and-toddler monitoring method includes: (a) a rear-seat infant-and-toddler monitoring device capturing video-and-audio signals directed at the child seat in the rear of the vehicle; (b) inputting the video-and-audio signals into an artificial intelligence engine to analyze the video for facial expressions of the infant and toddler and analyze the audio features to detect an infant emotional state, generating a notification event based on the infant current emotions; (c) the rear-seat infant-and-toddler monitoring device sending the notification event to a front-seat display device; (d) based on the notification event, the front-seat display device streams and retrieves the video-and-audio signals from a multimedia processing module and displays the video-and-audio; and (e) the front-seat display device notifying the rear-seat infant-and-toddler monitoring device based on the notification event, to request the streaming of video-and-audio packets to play the video-and-audio signals in the front-seat display device of the vehicle.

Description

Technical Field

[0001] This case relates to an in-vehicle passenger monitoring system and method, and more particularly to an in-vehicle infant and toddler monitoring system and method. Prior Technology

[0002] Infants and toddlers are small, fragile, and unable to care for themselves, requiring constant care from a caregiver. Their activity schedules are often unpredictable; sometimes caregivers interact with them, and other times they allow them time to do their own things. In this highly mobile environment, even a slight lapse in attention can lead to unfortunate incidents, making extra caution essential when caring for infants and toddlers.

[0003] The current approach involves a monitor setting up a camera next to the infant's bed or in their play area to record the infant's activities. Meanwhile, the user's mobile device is equipped with monitoring software that connects to the camera, using image recognition to assess the infant's activity in the footage. If any abnormality is detected, the mobile device immediately issues an alert, prompting the caregiver to drop everything and attend to the infant.

[0004] When caregivers are driving with infants or toddlers, regulations require the use of infant safety seats in the back seat to reduce the risk to infants and toddlers in car accidents and ensure their safety. When caregivers are driving alone with infants or toddlers, looking back while driving from the front seat may raise concerns about safe driving. When caregivers need to concentrate on driving, they cannot look back to check on the infant or toddler in the back seat. If an emergency occurs to the infant or toddler in the back seat, such as being strangled by the seatbelt and unable to speak, the caregiver cannot immediately take emergency measures.

[0005] Therefore, the ability for caregivers to monitor the condition of infants and toddlers in the back seat while driving is a noteworthy issue. Summary of the Invention

[0006] One embodiment of this invention is an in-vehicle infant monitoring method, suitable for execution by an in-vehicle infant monitoring system, which includes a rear-seat infant monitoring device and a front-seat display device coupled to the rear-seat infant monitoring device. The in-vehicle infant monitoring method includes (a) the audio-visual capture module of the rear-seat infant monitoring device takes pictures of the child seat in the rear seat of the vehicle to capture audio-visual signals; (b) the audio-visual processing module of the rear-seat infant monitoring device inputs the audio-visual signals into an artificial intelligence engine, which analyzes the images of the audio-visual signals, detects the infant's facial expressions, and analyzes the infant's vocal emotions from the voice characteristics, and generates a notification event corresponding to the infant's current emotions; (c) the rear-seat processing module of the rear-seat infant monitoring device receives the notification event and sends it to the front-seat display device; (d) according to the notification event, the audio-visual playback module of the front-seat display device streams and displays the audio-visual signals to the audio-visual processing module; and (e) the front-seat processing module of the front-seat display device notifies the audio-visual playback module to stream the audio-visual signals to play the infant's audio-visual signals in the front seat of the vehicle through the audio-visual playback module.

[0007] One embodiment of this invention is an in-vehicle infant monitoring system, including a rear-seat infant monitoring device and a front-seat display device coupled to the rear-seat infant monitoring device. The rear-seat infant monitoring device includes an audio / video capture module, an audio / video processing module, and a rear-seat processing module. The audio / video capture module captures audio / video signals by shooting towards the child seat in the rear of the vehicle. The audio / video processing module includes an artificial intelligence engine, which inputs the audio / video signals into the AI ​​engine to analyze the infant's facial expressions and vocal emotions based on voice characteristics, thereby generating a notification event corresponding to the infant's current emotion. The rear-seat processing module is coupled to the audio / video processing module and receives the notification event. The front-seat display device includes an audio / video playback module and a front-seat processing module. The audio / video playback module streams and displays audio / video signals to the audio / video processing module. The front seat processing module is coupled to the audio-visual playback module and notifies the audio-visual playback module to stream audio-visual content based on notification events, so that the audio-visual signals of infants and young children can be played in the front seat of the vehicle through the audio-visual playback module. Simple Explanation of the Diagram

[0008] Figure 1 is a block diagram of an in-vehicle infant monitoring system.

[0009] Figure 2 is a flowchart of the in-vehicle infant and toddler monitoring method. Implementation

[0010] The following description, in conjunction with the accompanying drawings and embodiments, further illustrates this case so that those skilled in the art can better understand and implement the invention. However, the embodiments described are not intended to limit the scope of the invention.

[0011] Considering that caregivers are usually drivers in the front seat, while infants and toddlers must sit in child safety seats in the back seat, caregivers or users cannot always keep an eye on the condition of infants and toddlers in the back seat. This case proposes a solution to the above situation.

[0012] Please refer to Figure 1, which is a block diagram of the in-vehicle infant monitoring system.

[0013] The in-vehicle infant monitoring system includes a rear-seat infant monitoring device 1 and a front-seat display device 2.

[0014] The rear-seat infant monitoring device 1 is mounted on the vehicle's ceiling, positioned to capture the face, voice, and environmental information of an infant in a child safety seat. For example, the rear-seat infant monitoring device 1 can be integrated into the rear-seat ceiling light fixture or placed near the light fixture with a bracket to ensure stable installation and prevent lens angle shift. Alternatively, a sliding rail can be installed on the ceiling to freely control the position of the rear-seat infant monitoring device 1.

[0015] The rear seat infant monitoring device 1 can be installed near the power supply unit in the rear seat to ensure a stable and uninterrupted power supply for the rear seat infant monitoring device 1.

[0016] The front seat display 2 is mounted in the front seat for caregivers or passengers in the front passenger seat to view. The front seat display 2 can be a stand-alone display, a handheld device (such as a mobile phone or tablet), or an in-vehicle infotainment system (IVI system).

[0017] The rear-seat infant monitoring device 1 is coupled to the front-seat display device 2. The rear-seat infant monitoring device 1 sends a notification to the front-seat display device 2 when it detects events involving the infant in the rear seat, alerting the caregiver to the infant's current condition. Detailed explanation follows.

[0018] The rear seat infant monitoring device 1 includes an audio-visual capture module 11, an audio-visual processing module 12, a rear seat processing module 13, a rear seat communication module 14, a temperature sensor 15, and an air quality sensor 16.

[0019] The audio / video capture module 11 is mounted on the vehicle ceiling and faces the child seat in the rear seat to capture the audio / video signals of infants and toddlers. The audio / video capture module 11 can be a camera. The audio / video capture module 11 can be a visible light and / or infrared light sensing module.

[0020] The audio-visual processing module 12 is coupled to the audio-visual capture module 11. The audio-visual processing module 12 includes an artificial intelligence engine 121, an encoder 123, and a streaming server 125.

[0021] The audio-visual processing module 12 inputs the video signal into the artificial intelligence engine 121, which analyzes the infant's facial expressions. Simultaneously, the audio-visual processing module 12 inputs the audio signal into the artificial intelligence engine 121, which analyzes the infant's vocal emotions based on vocal characteristics. The audio-visual processing module 12 then uses the analysis function of the artificial intelligence engine 121 to generate a notification event corresponding to the infant's current emotional state.

[0022] For example, the AI ​​engine 121 analyzes images and detects the infant's face, continuously tracking facial movements. If facial tracking is interrupted, such as disappearing from the image or being obscured by an unidentified object, the AI ​​engine 121 generates a notification event, such as when the infant slips out of the car seat or is strangled by the seatbelt. Furthermore, the AI ​​engine 121 analyzes images to detect the infant's facial expressions, such as sleeping, frowning, staring curiously, pouting, or smiling, identifying the infant's current emotional state. Caregivers can pre-set notification events to trigger when the infant displays specific emotions, allowing them to view or save video clips. The AI ​​engine 121 also analyzes images to detect abnormal infant behavior, such as prolonged inactivity, rolling over, or unusual twisting or contorting movements. The AI ​​engine 121 generates notification events to inform caregivers of the infant's current status in the back seat.

[0023] The artificial intelligence engine 121 can be an artificial intelligence model trained using various types of neural networks such as Artificial Neural Network (ANN), Deep Neural Network (DNN), Convolutional Neural Network (CNN), and Recurrent Neural Network (RNN), and can perform tasks such as infant facial expression recognition, upper body body language recognition, and speech emotion recognition.

[0024] The audio-visual processing module 12 can be a graphics processor, which performs image optimization processing on the images input from the audio-visual capture module 11, such as adjusting automatic exposure, automatic white balance, high dynamic range adjustment, and wide dynamic range adjustment. Since the vehicle is driving on the road, the light inside the car will vary with the changes in the external environment, affecting the filming inside the car. Therefore, the audio-visual processing module 12 optimizes the images to eliminate the brightness differences between each image, so as to improve the accuracy of image recognition or analysis performed by the artificial intelligence engine 121.

[0025] The rear-seat infant monitoring device 1 can optionally be equipped with an infrared supplementary lighting module (not shown). When the audio-visual processing module 12 detects insufficient light source, it triggers the infrared supplementary lighting module to emit infrared light, thereby improving the quality of the images captured by the audio-visual acquisition module 11.

[0026] In addition, vehicles traveling on the road are in areas with high environmental noise. The AI ​​engine 121 will use an AI noise reduction program to filter out external vehicle and environmental noise. The noise-reduced voice signal can improve the accuracy of the AI ​​engine 121 in analyzing the emotional state of infants' voices from the sound characteristics.

[0027] On the other hand, the AI ​​engine 121 analyzes audio signals to recognize the infant's expressions. For example, the AI ​​engine 121 uses an infant speech translation model (a pre-trained AI model) to identify the emotional meaning or intended words of the sounds the infant is making, so as to provide suggestions to caregivers later.

[0028] On the other hand, the encoder 123 of the audio-visual processing module 12 receives the audio-visual signals captured by the audio-visual capture module 11 through the artificial intelligence engine 121, and encodes and compresses the audio-visual signals to generate data in streaming packet format (such as HLS (HTTP Live Streaming), MPEG-DASH (Dynamic Adaptive Streaming over HTTP), RTMP (Real-Time Messaging Protocol) and RTSP (Real-Time Streaming Protocol)).

[0029] The streaming server 125 of the audio-visual processing module 12 uses the rear-seat infant monitoring device 1 as a streaming server to provide audio-visual signals to the front-seat display device 2 through the streaming service.

[0030] The back-end processing module 13 can be a system on a chip (SoC). The back-end processing module 13 is coupled to the audio-visual processing module 12 and the back-end communication module 14. The back-end processing module 13 is used for the initialization and setting of the back-end infant monitoring device 1, as well as the judgment and processing of notification events. When it receives a notification event from the audio-visual processing module 12, it sends a notification or alarm to the front-end display device 2 through the back-end communication module 14.

[0031] The rear seat processing module 13 also detects whether an infant or toddler has been left alone in the vehicle. For example, the rear seat processing module 13 detects that the vehicle has been turned off, but the audio-visual processing module 12 continues to detect the infant's face and / or voice, causing the rear seat processing module 13 to continuously receive notification events. At this time, the rear seat processing module 13 will issue an alarm signal, and the audio-visual playback module 21 of the front seat display device 2 will generate various types of reminders, such as warning sounds, flashing text on the locked screen, and flashing lights on the screen. The caregiver needs to click the confirmation button to end the reminder.

[0032] Temperature sensor 15 is coupled to rear seat processing module 13 and used to sense the interior temperature of the vehicle. If the rear seat processing module 13 determines that the interior temperature exceeds the comfortable temperature range (e.g., between 22°C and 24°C), it will send a notification event to the front seat display device 2 to remind the caregiver to adjust the interior air conditioning temperature to avoid heatstroke or catching a cold in infants and young children.

[0033] Air quality sensor 16 is coupled to rear seat processing module 13 and used to sense the air quality inside the vehicle. If the rear seat processing module 13 determines that the air quality inside the vehicle exceeds the comfort range (e.g., carbon dioxide concentration exceeds 1000 ppm or particulate matter PM2.5 exceeds 12 micrograms per cubic meter), it will send a notification event to front seat display device 2 to remind caregivers to adjust the air conditioning intensity so that infants and young children can breathe fresh air.

[0034] The front seat display device 2 includes an audio-visual playback module 21, a front seat processing module 22, a storage module 23, and a front seat communication module 24.

[0035] The front seat communication module 24 of the front seat display device 2 is connected to the rear seat communication module 14 of the rear seat infant monitoring device 1. The front seat communication module 24 and the rear seat communication module 14 can be any wireless communication connection method, such as Wi-Fi or Bluetooth, or a wired communication connection method, such as Ethernet or USB.

[0036] The audio / video playback module 21 is used to play audio / video signals from the rear-seat infant monitoring device 1. The audio / video playback module 21 includes a streaming client 211, a decoder 213, and a display module 215.

[0037] The streaming client 211 is a client that uses the front-seat display device 2 as a streaming service to receive audio and video signals through the streaming service of the rear-seat infant monitoring device 1.

[0038] Decoder 213 decodes the received audio and video signals into images and sounds, and provides the decoded audio and video signals to display module 215 for playback.

[0039] The front-seat processing module 22 is coupled to the audio / video playback module 21. The front-seat processing module 22 receives notification events from the rear-seat infant monitoring device 1 via the front-seat communication module 24, and displays warning messages on the display module 215 or plays warning sounds on the audio playback module (not shown). The front-seat processing module 22 can be a system on a chip (SoC).

[0040] Storage module 23 is coupled to front-seat processing module 22 and front-seat communication module 24 to store audio and video signals from rear-seat infant monitoring device 1. Storage module 23 stores the audio and video signal of each notification event as a record of each notification event. For example, if the infant's expression is a happy laugh, after the rear-seat infant monitoring device 1 triggers a notification event, the audio and video signal is streamed to the front-seat display device 2, played on audio and video playback module 21, and stored in storage module 23.

[0041] Figure 2 is a flowchart of the in-vehicle infant monitoring method. The in-vehicle infant monitoring method can be executed by the in-vehicle infant monitoring system shown in Figure 1.

[0042] Step S31: The audio-visual capture module 11 shoots towards the child seat in the back of the vehicle to capture audio-visual signals.

[0043] In step S32, the audio-visual processing module 12 inputs the audio-visual signal into the artificial intelligence engine 121.

[0044] In step S33, the artificial intelligence engine 121 analyzes the infant's facial expressions and vocal emotions from the vocal features, and generates a notification event corresponding to the infant's current emotion.

[0045] Step S34: The rear seat processing module 13 receives a notification event.

[0046] In step S35, the streaming client 211 of the audio-visual playback module 21 obtains and displays audio-visual signals from the streaming server 125 of the audio-visual processing module 12.

[0047] In step S36, the front seat processing module 22 notifies the audio-visual playback module 21 to stream audio-visual content according to the notification event, so as to play the audio-visual signal of the infant in the front seat of the vehicle through the audio-visual playback module 21.

[0048] In summary, the in-vehicle infant monitoring system and method described in this case are applied within the vehicle environment to monitor the status of infants and toddlers seated in the back seat and provide real-time notifications. Whether the infant or toddler is in an abnormal state (requiring close attention from the caregiver) or a normal state (notifying the caregiver to share in the infant's happiness and enjoy their cuteness), the caregiver can fully monitor the infant's status without turning around. This allows caregivers to balance public safety while driving with humanitarian care and protection of the infant's life, enhancing both safety measures.

[0049] The above description is merely a specific example of this case and does not limit the scope of the patent application in this case. Therefore, all equivalent changes made using the content of this case are similarly included within the scope of this case and are hereby stated.

[0050] 1: Back seat infant monitoring device 11: Audio / Video Capture Module 12: Audio / Video Processing Module 121: Artificial Intelligence Engine 123: Encoder 125: Streaming Servo 13: Rear seat handling module 14: Rear seat communication module 15: Temperature sensor 16: Air Quality Sensor 2: Front seat display device 21: Audio / Video Playback Module 211: Streaming Client 213: Decoder 215: Display Module 22: Front seat processing module 23: Storage Module 24: Front seat communication module S31~S36: Steps

Claims

1. A method for monitoring infants and toddlers inside a vehicle, suitable for execution by an in-vehicle infant and toddler monitoring system, the in-vehicle infant and toddler monitoring system comprising a rear-seat infant and toddler monitoring device and a front-seat display device coupled to the rear-seat infant and toddler monitoring device, wherein the in-vehicle infant and toddler monitoring method comprises: Step (a) An audio-visual capture module of the rear-seat infant monitoring device captures an audio-visual signal by shooting towards a child seat in the rear of the vehicle; Step (b) An audio-visual processing module of the rear-seat infant monitoring device inputs the audio-visual signal into an artificial intelligence engine. The artificial intelligence engine analyzes the image of the audio-visual signal, detects the infant's facial expressions, analyzes the infant's vocal emotions from sound characteristics, or detects whether there are any abnormalities in the infant's facial, head, or posture behavior, and generates a notification event corresponding to the infant's current emotion or abnormal behavior; Step (c) An encoder of the rear-seat infant monitoring device encodes and compresses the audio-visual signal to generate a streaming audio-visual signal in a streaming packet format; Step (d) A streaming server of the rear-seat infant monitoring device provides the streaming audio-visual signal via a streaming service; Step (e) A rear-seat processing module of the rear-seat infant monitoring device receives the notification event and sends it to the front-seat display device; Step (f) In response to the notification event, a streaming client of the front seat display device obtains the streaming audio and video from the streaming server via the streaming service; step (g) a decoder of the front seat display device decodes the streaming audio and video; and step (h) a display module of the front seat display device plays the decoded streaming audio and video to play an infant's audio and video in the front seat of the vehicle.

2. The in-vehicle infant monitoring method as described in claim 1, wherein step (b) includes: Step (b11) The artificial intelligence engine detects the infant's face and continuously tracks the movement of the infant's face; And in step (b12), when the AI ​​engine detects that the infant's face has disappeared from the image or is obscured by an unidentified object, the AI ​​engine generates the notification event.

3. The in-vehicle infant monitoring method as described in claim 1, wherein the notification event includes the infant slipping out of the child safety seat, having their head strangled by the seat belt, falling asleep, frowning, looking around curiously with wide eyes, pouting, smiling, remaining inactive for a long time, or rolling over or twisting in strange postures.

4. The in-vehicle infant monitoring method as described in claim 1, wherein step (e) includes: Step (e1) When the rear seat processing module receives the notification event that an infant or toddler has been left alone in the vehicle, the rear seat processing module sends an alert signal to the front seat display device; and Step (e2) The audio-visual playback module of the front seat display device generates multiple types of alerts, including at least one of an alert sound, flashing text on a locked screen, and flashing screen lights.

5. The in-vehicle infant monitoring method as described in claim 1 further includes: The rear-seat infant monitoring device has a temperature sensor that senses the interior temperature. When the rear-seat processing module determines that the interior temperature exceeds the comfortable temperature range, it sends a notification event to the front-seat display device. The rear-seat infant monitoring device also has an air quality sensor that senses the interior air quality. When the rear-seat processing module determines that the interior air quality exceeds the comfortable range, it sends a notification event to the front-seat display device.

6. An in-vehicle infant monitoring system, comprising: A rear-seat infant monitoring device includes: an audio / video capture module that captures an audio / video signal by shooting towards a child seat in the rear seat of a vehicle; an audio / video processing module including an artificial intelligence engine, an encoder, and a streaming server, wherein the audio / video processing module inputs the audio / video signal into the artificial intelligence engine to analyze the infant's facial expressions and analyze the infant's vocal emotions from voice characteristics, or detect whether there are any abnormalities in the infant's facial, head, or postural behaviors, so as to generate a notification event corresponding to the infant's current emotions or abnormal behaviors; the encoder encodes and compresses the audio / video signal to generate a streaming audio / video in a streaming packet format; the streaming server is used to provide the streaming audio / video via a streaming service; and a rear-seat processing module coupled to the audio / video processing module and receiving the notification event; and a front-seat display device coupled to the rear-seat infant monitoring device, including: An audio-visual playback module includes a streaming client, a decoder, and a display module. The streaming client is used to obtain the streaming audio-visual content from the streaming server via the streaming service in response to the notification event. The decoder is used to decode the streaming audio-visual content. The display module is used to play the decoded streaming audio-visual content for playing audio-visual content for infants and young children in the front seat of a vehicle. A front seat processing module is coupled to the audio-visual playback module and notifies the audio-visual playback module to perform streaming audio-visual playback according to the notification event.

7. The in-vehicle infant monitoring system as described in claim 6, wherein the artificial intelligence engine detects the infant's face and continuously tracks the movement of the infant's face, and generates the notification event when the infant's face is detected to disappear from the image of the audio-visual signal or be obscured by an unidentified object.

8. The in-vehicle infant monitoring system as described in claim 6, wherein the notification events include the infant slipping out of the child safety seat, having their head caught in the seatbelt, falling asleep, frowning, looking around curiously with wide eyes, pouting, smiling, remaining inactive for a long time, or rolling over or twisting in strange postures.

9. The in-vehicle infant monitoring system as claimed in claim 6, wherein when the rear seat processing module receives the notification event that an infant has been left alone in the vehicle, the rear seat processing module sends an alert signal to the front seat display device; and the audio-visual playback module of the front seat display device generates multiple types of alerts, including at least one of an alert sound, flashing text on a locked screen, and flashing screen lights.

10. The in-vehicle infant monitoring system as claimed in claim 6, wherein the rear-seat infant monitoring device further includes: A temperature sensor is coupled to the rear seat processing module and senses the temperature inside the vehicle; An air quality sensor is coupled to the rear seat processing module and senses the air quality inside the vehicle; wherein the rear seat processing module sends a notification event to the front seat display device when it determines that the temperature inside the vehicle exceeds the comfortable temperature range, and sends a notification event to the front seat display device when it determines that the air quality inside the vehicle exceeds the comfortable range.

Images