Occupant monitoring system for vehicle

Abstract

An occupant monitoring system for monitoring one or more occupants located on a seat of a vehicle includes one or more cameras positioned as cameras that capture image data representative of the occupant, one or more biometric sensors positioned as cameras that capture image data representative of the occupant, and one or more controllers positioned as cameras that capture image data representative of the occupant. Each of the one or more biometric sensors monitors a parameter indicative of a physiological metric of the occupant. The one or more controllers execute instructions to determine one or more physical signs of the occupant based on image data collected by the one or more cameras and one or more parameters indicative of the one or more physical signs of each occupant collected by the one or more biometric sensors, and in response to determining that the physiological metric exceeds the primary event threshold or the secondary event threshold, generating a notification for the driver of the vehicle.

Description

Technical Field

[0001] The present disclosure relates to an occupant monitoring system that monitors one or more physiological indicators of one or more occupants located in a vehicle seat. Background Art

[0002] Children twelve years of age and younger, or below the minimum height standard, should generally be placed in one of the vehicle's rear seats. In many cases, the caregiver of a child placed in a rear seat is driving the vehicle as the driver. Therefore, the caregiver may have to turn their head away from the road they are viewing through the windshield to see the child sitting in the back seat of the vehicle. As a result, it may be difficult for the caregiver to focus or care for the child while driving the vehicle. For example, very young children, such as infants or toddlers, may become restless, agitated, or have tantrums while riding in a vehicle. It may be difficult for a caregiver to calm a toddler while driving the vehicle. Additionally, it is important to note that other people, such as very young children and the elderly, may not be able to verbally express their health concerns or when they are feeling unwell. This may also be problematic because the caregiver cannot check on or care for these people while driving the vehicle.

[0003] Therefore, while current vehicles achieve their intended purpose, there remains a need in the art for an improved method of monitoring rear seat occupants by a vehicle operator. Summary of the invention

[0004] According to several aspects, the present invention discloses an occupant monitoring system for monitoring one or more occupants located in a vehicle seat. The occupant monitoring system includes: one or more cameras, one or more biometric sensors, and one or more controllers, wherein the one or more cameras are positioned to capture image data representing the occupants, each of the one or more biometric sensors monitors a parameter indicating a physiological indicator of the occupant, and the one or more controllers are in electronic communication with the one or more cameras and the one or more biometric sensors. The one or more controllers execute instructions to determine one or more physiological indicators of the occupants based on the image data collected by the one or more cameras and the one or more parameters indicating one or more physiological indicators of each occupant collected by the one or more biometric sensors. The one or more controllers compare the physiological indicators of the occupants with a baseline value, wherein the baseline value of each physiological indicator indicates that the occupant is functioning normally. In response to determining that the physiological indicator falls outside an allowable range of the baseline value, the one or more controllers compare the physiological indicator with at least one of a major event threshold and a minor event threshold. In response to determining that the physiological indicator exceeds the major event threshold or the minor event threshold, the one or more controllers generate a notification for the vehicle driver.

[0005] In another aspect, one or more controllers execute instructions to determine that a physiological indicator exceeds a major event threshold, and in response to determining that the physiological indicator exceeds the major event threshold, determine that immediate action by the occupant is required.

[0006] In yet another aspect, the notification includes a visual alert and a voice alert.

[0007] In one aspect, one or more controllers execute instructions to direct the vehicle to perform one or more of the following corrective actions: switch to a fully autonomous driving mode so that the driver can care for the occupants, navigate the vehicle to the nearest hospital, contact emergency personnel via an onboard wireless communication system, and perform an automatic pull-over via the automated driving system.

[0008] In another aspect, one or more controllers execute instructions to determine that the physiological indicator exceeds a minor event threshold, and in response to determining that the physiological indicator exceeds the minor event threshold, determine that the occupant is experiencing a physical problem that requires monitoring by a vehicle driver.

[0009] In another aspect, the notification includes a visual message notifying the driver that the physiological indicator has exceeded the minor event threshold.

[0010] In yet another aspect, the occupant monitoring system further includes a front row display in electronic communication with the one or more controllers, wherein the front row display generates visual content visible to a driver of the vehicle.

[0011] In one aspect, the front display is one of: an augmented reality head-up display (AR-HUD) and an infotainment screen.

[0012] In another aspect, one or more controllers execute instructions to direct the front row display to generate visual content representative of a physiological indicator of each of the one or more occupants.

[0013] In yet another aspect, the visual content includes a real-time video feed based on image data captured by one or more cameras.

[0014] In one aspect, the occupant monitoring system also includes one or more occupant databases in electronic communication with the one or more controllers, the occupant databases storing facial recognition data representing facial features of one or more known individuals and storing data indicating baseline values ​​of physiological indicators corresponding to specific known individuals.

[0015] In another aspect, one or more controllers execute instructions to perform one or more facial recognition algorithms to determine an identity of an occupant based on image data captured by one or more cameras, and to compare the identity of the occupant to facial recognition data of one or more known individuals stored in one or more occupant databases.

[0016] In yet another aspect, in response to determining that there is a match between the identity of the occupant and facial recognition data representing one of the known individuals, the one or more controllers utilize a baseline value for a physiological indicator corresponding to the particular known individual stored in the one or more occupant database.

[0017] In one aspect, in response to determining that there is no match, the one or more controllers utilize a default baseline value for the physiological indicator.

[0018] In another aspect, the one or more cameras include one or more of: a red, green, and blue (RGB) camera that captures visible light image data, an infrared camera that captures infrared image data, and a thermal camera that captures thermal image data.

[0019] In yet another aspect, the physiological indicator comprises one or more of: heart rate, breathing rate, body temperature, muscle stiffness, sweat rate, sleepiness, stress, motion sickness, pupil dilation, blood sugar level, and blood pressure.

[0020] In one aspect, the occupant monitoring system further includes one or more radar sensors in electronic communication with the one or more controllers, wherein the one or more radar sensors generate signals indicative of the presence and seat position of the occupant.

[0021] In another aspect, the present invention discloses a method for monitoring one or more occupants located in vehicle seats by an occupant monitoring system. The method includes determining, by one or more controllers, one or more physiological indicators of the occupants based on image data collected by one or more cameras and one or more parameters indicating one or more physiological indicators of each occupant collected by one or more biometric sensors. The method includes comparing, by one or more controllers, the physiological indicators of the occupants with baseline values, wherein the baseline value of each physiological indicator indicates that the occupant is functioning normally. In response to determining that the physiological indicator falls outside an allowable range of the baseline value, the method includes comparing the physiological indicator with a major event threshold and a minor event threshold. In response to determining that the physiological indicator exceeds a major event threshold or a minor event threshold, the method includes generating a notification for a vehicle driver.

[0022] In another aspect, the present invention discloses an occupant monitoring system for monitoring one or more occupants located in a vehicle seat. The occupant monitoring system includes: one or more cameras, one or more biometric sensors, and one or more controllers, wherein the one or more cameras are positioned to capture image data representing the occupants, each of the one or more sensors monitors a parameter indicating a physiological indicator of the one or more occupants, and the one or more controllers are in electronic communication with the one or more cameras and the one or more biometric sensors. The one or more controllers execute instructions to determine one or more physiological indicators of the occupants based on the image data collected by the one or more cameras and the one or more parameters indicating one or more physiological indicators of each occupant collected by the one or more biometric sensors. The one or more controllers compare the physiological indicators of the occupants with a baseline value, wherein the baseline value of each physiological indicator indicates that the occupant is functioning normally. In response to determining that the physiological indicator falls outside the allowable range of the baseline value, the one or more controllers compare the physiological indicator with a major event threshold. In response to determining that the physiological indicator exceeds the major event threshold, the one or more controllers generate a notification for the vehicle driver and determine that the occupant needs to take immediate action.

[0023] In another aspect, in response to determining that the physiological indicator does not exceed the primary event threshold, the one or more controllers compare the physiological indicator to a secondary event threshold, determine that the physiological indicator exceeds the secondary event threshold, and in response to determining that the physiological indicator exceeds the secondary event threshold, determine that the occupant is experiencing a physical problem that requires monitoring by the vehicle driver.

[0024] Further areas of applicability will become apparent from the description provided herein.It should be understood that the description and specific examples are intended for purposes of illustration only and are not intended to limit the scope of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings described herein are for illustration purposes only and are not intended to limit the scope of the present disclosure in any way.

[0026] Figure 1 is a schematic diagram of a vehicle including the disclosed occupant monitoring system for monitoring one or more physiological indicators of one or more occupants according to an exemplary embodiment;

[0027] Figure 2A is an exemplary illustration of an infotainment screen of an occupant monitoring system according to an exemplary embodiment, wherein the infotainment screen displays visual content representative of a physiological indicator of each occupant located in a vehicle;

[0028] Figure 2B According to an exemplary embodiment, Figure 2AAn exemplary illustration of another embodiment of an infotainment screen is shown, wherein the infotainment screen displays physiological indicators for each occupant based on seat position;

[0029] Figure 2C According to an exemplary embodiment, Figure 2A An exemplary illustration of another embodiment of an infotainment screen is shown, wherein the infotainment screen displays a real-time video feed of image data representative of one or more occupants; and

[0030] Figure 3 According to an exemplary embodiment, Figure 1 A process flow diagram is shown that illustrates a method for monitoring one or more occupants located in a vehicle by the disclosed occupant monitoring system. DETAILED DESCRIPTION

[0031] The following description is merely exemplary in nature and is not intended to limit the present disclosure, application, or uses.

[0032] refer to Figure 1 , an occupant monitoring system 10 for a vehicle 12 is shown. As explained below, the occupant monitoring system 10 monitors one or more physiological indicators of one or more occupants 14 located in one of the seats 16 of the vehicle 12. In a non-limiting embodiment, the seat 16 is part of the second row of seats, the third row of seats, or the fourth row of seats in the vehicle 12. In an embodiment, the monitored one or more occupants 14 are located in the interior cab 18 of the vehicle 12, but are not sitting in any seat in the interior cab 18. In one embodiment, the one or more occupants 14 include individuals who need close attention, such as infants, toddlers, young children under 12 years old, or very elderly people. In an alternative, the one or more occupants 14 may include animals, such as pets such as dogs or cats. It should be understood that the vehicle 12 can be any type of vehicle, such as, but not limited to, a car, a truck, a sport utility vehicle, a van, a bus, or an RV.

[0033] The occupant monitoring system 10 includes one or more controllers 20 in electronic communication with a front display 22, a speaker 24, one or more radar sensors 26, one or more cameras 28, and one or more biometric sensors 30. In one embodiment, the one or more controllers 20 are in wireless communication with a personal electronic device 32 that is associated with a driver of the vehicle 12. The personal electronic device 32 is any type of portable electronic device, such as a smart phone or tablet computer, that has wireless capabilities and includes one or more processors and a display 36 that displays images and graphics to the driver.

[0034] The front display 22 is located within the interior cabin 18 of the vehicle 12 and generates visual content (e.g., graphics and text) that is visible to the driver of the vehicle 12. It should be appreciated that the visual content generated by the front display 22 is visible while the driver is driving the vehicle 12 without requiring the driver to turn his or her head or body toward the rear of the vehicle 12. In one non-limiting embodiment, the front display 22 is an augmented reality head-up display (AR-HUD) that generates visual content directly on the windshield of the vehicle 12 in the driver's forward field of view. Alternatively, in another embodiment, the front display 22 includes an infotainment screen 34 ( Figure 2A ), wherein the infotainment screen 34 is mounted to a dashboard 38 of the vehicle 12 .

[0035] The one or more radar sensors 26 provide signals to the one or more controllers 20 indicating the presence and seat position of the occupant 14. The one or more cameras 28 are located within the interior cabin 18 of the vehicle 12 and are positioned to capture image data representing the occupant 14 located in the seat 16. The one or more cameras 28 include a red, green, and blue (RGB) camera that captures visible light image data, an infrared camera that captures infrared image data, or a thermal camera that captures thermal image data. It should be understood that the image data includes visible light image data, infrared image data, and thermal image data.

[0036] The one or more biometric sensors 30 each monitor a parameter indicating a physiological indicator of an occupant 14 located in a seat 16 of the vehicle 12. The physiological indicator represents the physical function of an individual. Some examples of physiological indicators of the occupant 14 include, but are not limited to, heart rate, breathing rate, body temperature, muscle stiffness, sweat rate, drowsiness, stress, motion sickness, pupil dilation, blood sugar level, and blood pressure. It should be understood that in an embodiment, the one or more biometric sensors 30 include one or more radar sensors 26 and one or more cameras 28. Some examples of the one or more biometric sensors 30 include, but are not limited to, a heart rate sensor, a galvanic skin response (GSR) sensor that detects the conductivity of the skin of the occupant 14, and a piezoelectric muscle sensor that detects the piezoelectric resonance of the occupant 14, wherein the piezoelectric resonance indicates muscle stiffness. In an embodiment, the GSR sensor is mounted to the surface of a door handle or an armrest to contact the skin of the occupant 14.

[0037] In an embodiment, the one or more controllers 20 determine the sweat rate of the occupant 14 based on thermal image data representing the face of the occupant 14, and specifically the forehead of the occupant 14, the thermal image data being captured by the one or more thermal image cameras 28. In an embodiment, the sweat rate of the occupant 14 is further determined based on the visible light image data captured by the one or more RGB cameras 28.

[0038] In an embodiment, the sleepiness of the occupant 14 is determined based on image data captured by the one or more cameras 28, the heart rate of the occupant detected by the heart rate sensor, and the conductivity of the skin of the occupant 14 measured by the GSR sensor. For example, the image data captured by the one or more cameras 28 indicates various behaviors of the occupant 14 that indicate sleepiness, such as, but not limited to, yawning, closing eyes, blinking, and head posture. In addition, it should also be understood that the heart rate of the occupant 14 varies significantly between different stages of sleepiness, such as alertness and fatigue.

[0039] In one embodiment, the one or more controllers 20 determine the stress of the occupant 14 based on the heart rate of the occupant 14 detected by the heart rate sensor and the conductivity of the skin of the occupant 14 measured by the GSR sensor. Alternatively, in another embodiment, the one or more controllers 20 also determine the stress of the occupant 14 based on image data collected by the one or more cameras 28. In an embodiment, motion sickness of the occupant 14 is determined based on pupil rhythm monitored by the image data captured by the one or more cameras 28.

[0040] Continue to refer Figure 1 During initialization of the occupant monitoring system 10, the one or more controllers 20 monitor image data representing an occupant 14 located in the seat 16 captured by the one or more cameras 28. The one or more controllers 20 execute one or more facial recognition algorithms to determine the identity of the occupant 14 based on the image data representing the occupant 14. The one or more controllers 20 are in electronic communication with one or more occupant databases 44, which store facial recognition data representing facial features of one or more known individuals. The one or more occupant databases 44 also store data indicating baseline values ​​of physiological indicators corresponding to specific known individuals. In an embodiment, the occupant database 44 is stored locally, however, the occupant database 44 can also be stored remotely on a network server.

[0041] The one or more controllers 20 execute instructions to compare the identity of the occupant 14 to the facial recognition data stored in the one or more occupant databases 44. In response to a match between the identified identity of the occupant 14 and one of the known individuals stored in the occupant databases 44, the one or more controllers 20 utilize a baseline value for the physiological indicator corresponding to the particular known individual stored in the one or more occupant databases 44. However, if no match is identified, the one or more controllers 20 utilize a default baseline value for the physiological indicator of the occupant 14.

[0042] The one or more controllers 20 execute instructions to determine one or more physiological indicators of each occupant 14 based on one or more of the following: signals received from the one or more radar sensors 26, image data collected by the one or more cameras 28, and one or more parameters indicative of the one or more physiological indicators of each occupant 14 collected by the one or more biometric sensors 30. The one or more controllers 20 instruct the front display 22 to generate visual content representative of the physiological indicator of each occupant 14. Figure 2A is an exemplary illustration of a front row display 22, wherein the front row display 22 includes an infotainment screen 34 that displays visual content representative of a physiological indicator of each occupant 14 located in the vehicle 12. Figure 2A In the example shown, the infotainment screen 34 displays an icon 50 corresponding to one of the occupants 14 and one or more icons 52 representing physiological indicators corresponding to each of the occupants 14 located within the interior cabin 18 of the vehicle 12 .

[0043] In such Figure 2A In the example shown, the icon 50 represents the gender of the corresponding occupant 14, and one or more icons 52 represent the heart rate and breathing rate corresponding to each occupant 14 in the vehicle 12. However, it should be understood that the driver or another occupant may customize the front display 22 so that the icons 52 instead represent different or additional physiological indicators.

[0044] Figure 2B Yes Figure 2A An alternative embodiment of the infotainment screen 34 is shown, wherein the infotainment screen 34 displays physiological indicators of each occupant 14 based on the seat position. Figure 2B In the illustrated example, a rear passenger-side seating position 60 and a rear driver-side seating position 62 are shown, wherein each seating position 60 , 62 is associated with one or more icons 52 representing physiological indicators corresponding to the respective occupant 14 .

[0045] Figure 2C Yes Figure 2A Another embodiment of the infotainment screen 34 is shown, wherein the infotainment screen 34 is based on one or more cameras 28 ( Figure 1 ) captured image data displays a real-time video feed 64. The infotainment screen 34 also displays an icon 50 corresponding to one of the selected occupants 14 and one or more icons 52 representing the physiological indicators of the selected occupant 14. It should be understood that the driver can use a touch screen feature, a button, or based on an automatic switching feature to switch the view of the infotainment screen 34 to display the physiological indicators of one of the remaining occupants 14, as applicable.

[0046] refer to Figure 1, one or more controllers 20 execute instructions to continuously compare each physiological indicator of the occupant 14 with a baseline value. The baseline value corresponds to a specific physiological indicator and indicates that the occupant 14 is functioning normally and has not experienced health-related events. That is, in other words, the baseline value corresponding to the specific physiological indicator indicates that the occupant 14 does not have a physiological problem that requires immediate action or needs to be monitored by the driver. Some examples of physiological problems that need to be monitored by the driver include increased heart rate, increased body temperature, and high stress. It should be understood that physiological problems that need to be monitored by the driver do not require actual intervention. On the contrary, sometimes the occupant 14 may encounter physiological problems that require intervention by the driver, such as a heart attack, an anxiety attack, or an asthma attack. In an embodiment, the baseline value represents the allowable range of the corresponding physiological indicator, which indicates that the individual is functioning normally.

[0047] In response to determining that the physiological indicator falls outside of the allowed range of baseline values, the one or more controllers 20 compare the physiological indicator to a major event threshold. The one or more controllers 20 also compare the physiological indicator to a minor event threshold. The major event threshold indicates that the occupant 14 is experiencing a physiological problem that requires immediate action by the driver of the vehicle 12. In response to determining that the physiological indicator exceeds the major event threshold, the one or more controllers 20 determine that the occupant 14 needs to take immediate action. Some examples of physiological indicators represented by the major event threshold include, but are not limited to, a high fever (greater than about 39.5°C or 103°F), or a relatively sharp drop in blood pressure (a drop in blood pressure of at least 20 millimeters of mercury (mmHg)).

[0048] In response to determining that the occupant 14 needs to take immediate action, the one or more controllers 20 generate a notification for the driver of the vehicle 12. In one embodiment, the notification includes a visual alert created by the front display 22 and a voice alert generated by the speaker 24. For example, the visual alert may include an emergency graphic and a message indicating that the occupant 14 needs to take immediate action, and the voice alert may include an alarm, tone, or voice message indicating that immediate action is required. In one embodiment, the one or more controllers 20 may transmit a signal indicating that the personal electronic device 32 associated with the driver of the vehicle 12 also generates a visual alert, a voice alert, or both. It should be understood that the personal electronic device 32 can be used in the event that the driver temporarily leaves the vehicle 12.

[0049] In an embodiment, in response to determining that the occupant 14 requires immediate action, the one or more controllers 20 instruct the vehicle 12 to perform one or more of the following corrective actions: switch to a fully autonomous driving mode so that the driver can take care of the occupant 14, navigate the vehicle 12 to the nearest hospital or emergency room, contact emergency personnel via the onboard wireless communication system, and perform an automatic pullover via the automated driving system. In an embodiment, the one or more controllers 20 may instruct the front display 22 to generate a graphic requesting the driver of the vehicle 12 to select one of the corrective actions.

[0050] The minor event threshold indicates that the occupant 14 is experiencing a physiological problem that requires monitoring by the driver of the vehicle 12 but does not require immediate action. In response to determining that the physiological indicator exceeds the minor event threshold, the one or more controllers 20 generate a notification. When the occupant 14 does not require immediate attention, in one embodiment, the notification includes a visual message generated by the front display 22 that notifies the driver of the physiological indicator of the occupant 14. Figure 2C As shown, in an embodiment, the front display 22 generates an icon 50 corresponding to the occupant 14 and an icon 52 representing a physiological indicator. In an embodiment, the front display 22 also shows one or more cameras 28 ( Figure 1 ) real-time video transmission of the image data captured 64. Some example values ​​of physiological indicators represented by the minor event thresholds include, but are not limited to, low-grade fever (ranging between 37.7°C and 39.5°C or 100°F-103°F).

[0051] In one embodiment, the one or more controllers 20 execute one or more object detection and tracking algorithms to identify an object of interest within the control of the occupant 14, track the object of interest, and determine when the occupant 14 loses control of the object of interest based on image data captured by the one or more cameras 28. For example, in one embodiment, the occupant 14 is a toddler and the object of interest is a favorite toy or pacifier. The one or more controllers 20 analyze the image data captured by the one or more cameras 28 to identify the object of interest located in one or more hands or on the lap of the occupant 14 based on the one or more object detection algorithms. The one or more controllers 20 execute the tracking algorithm to continuously track the object of interest. In response to detecting that the object of interest has fallen to the floor of the interior cab 18 of the vehicle 12, the one or more controllers 20 determine that the occupant 14 has lost control of the object of interest. The one or more controllers 20 may also implement vomit detection, rear seat belt status monitoring, and determine whether the occupant 14 has passed a body part through a window of the vehicle 12 based on the one or more object detection and tracking algorithms.

[0052] In another embodiment, the one or more controllers 20 monitor signals received from the one or more radar sensors 26, image data collected by the one or more cameras 28, and one or more parameters indicating one or more physiological indicators of each occupant 14 collected by the one or more biometric sensors 30 to determine the comfort level of the occupant 14. The comfort level of the occupant 14 is determined based on the breathing rate and sweat rate of the occupant 14. It should be understood that in some cases, the driver may temporarily leave the vehicle 12, but the occupant 14 is still in the seat 16. The one or more controllers 20 compare the comfort level with a breathing rate threshold and a sweat rate threshold, wherein the breathing rate threshold and the sweat rate threshold indicate the discomfort of the occupant. In an embodiment, in response to determining that the comfort level exceeds the breathing rate threshold or the sweat rate threshold, the one or more controllers 20 instruct the in-cab heating, ventilation and air conditioning (HVAC) system to adjust the in-cab temperature of the interior cab 18. For example, in warmer temperatures, the in-cab HVAC adds cooling to the occupant 14, and in colder temperatures, the in-cab HVAC adds heating to the occupant 14. In one embodiment, one or more controllers 20 may transmit a signal instructing a personal electronic device 32 associated with a driver of the vehicle 12 to also generate a visual alert, an audible alert, or both indicating that the occupant 14 is experiencing discomfort.

[0053] Figure 3 is a process flow diagram illustrating a method 300 for monitoring one or more occupants 14 located in seats 16 of a vehicle 12 via an occupant monitoring system 10 . Figure 1 and Figure 3 , the method 300 may begin at block 302. In block 302, the one or more controllers 20 monitor image data captured by the one or more cameras 28 representing the occupants 14 located in the seats 16 of the vehicle 12. The method 300 may then proceed to block 302.

[0054] In block 302 , the one or more controllers 20 execute one or more facial recognition algorithms to determine the identity of the occupant 14 based on the image data. The method 300 may then proceed to decision block 304 .

[0055] In decision block 306, the one or more controllers 20 compare the identity of the occupant 14 to the facial recognition data of one or more known individuals stored in the one or more occupant databases 44 to determine whether a match exists between the identity of the occupant 14 and the facial recognition data representing one of the known individuals. In response to determining that a match exists, the method proceeds to block 308. In block 308, the one or more controllers 20 utilize the baseline value of the physiological indicator corresponding to the particular known individual stored in the one or more occupant databases 44, and the method 300 proceeds to block 312.

[0056] In response to determining that there is no match, the method proceeds to block 310. In block 310, the one or more controllers 20 utilize a default baseline value for the physiological indicator. The method 300 may then proceed to block 312.

[0057] In block 312 , the one or more controllers 20 execute instructions to determine one or more physiological indicators of the occupants 14 based on one or more of the following: signals received from the one or more radar sensors 26 , image data collected by the one or more cameras 28 , and one or more parameters collected by the one or more biometric sensors 30 that indicate one or more physiological indicators of each occupant 14 . The method 300 may then proceed to block 314 .

[0058] In block 314, the one or more controllers 20 instruct the front row displays 22 to generate visual content representative of the physiological indicators of each occupant 14, such as Figure 2A-2C The method 300 may then proceed to decision block 316 .

[0059] In decision block 316, the one or more controllers 20 compare the physiological indicator of the occupant 14 to a baseline value, wherein the baseline value of the physiological indicator indicates that the occupant is functioning normally. In response to determining that the physiological indicator falls outside the allowable range of the baseline value, the method 300 proceeds to decision block 318. Otherwise, the method 300 returns to block 312.

[0060] In decision block 318, the one or more controllers 20 compare the physiological indicator to the major event threshold. In response to determining that the physiological indicator exceeds the major event threshold, the one or more controllers 20 determine that the occupant 14 needs to take immediate action, and the method proceeds to block 320. Otherwise, the method 300 may proceed to decision block 324.

[0061] In block 320, in response to determining that the occupant 14 requires immediate action, the one or more controllers 20 generate a notification to the driver of the vehicle 12. As described above, in one embodiment, the notification includes a visual alert created by the front display 22 and a voice alert generated by the speaker 24. The method 300 may then proceed to block 322.

[0062] In block 322, the one or more controllers 20 instruct the vehicle 12 to perform one or more of the following corrective actions: switching to a fully autonomous driving mode so that the driver can take care of the occupants 14, navigating the vehicle 12 to the nearest hospital, contacting emergency personnel via the onboard wireless communication system, and performing an automatic pullover by the automated driving system. As described above, in an embodiment, the one or more controllers 20 may instruct the front display 22 to generate a graphic asking the driver of the vehicle 12 to select one of the corrective actions. The method 300 may then terminate or return to block 312.

[0063] In decision block 324, the one or more controllers 20 compare the physiological indicator to the minor event threshold. In response to determining that the physiological indicator exceeds the minor event threshold, the one or more controllers 20 determine that the occupant 14 is experiencing a physical problem that needs to be monitored by the driver of the vehicle 12, and the method 300 proceeds to block 326. Otherwise, the method 300 returns to block 312.

[0064] In block 326, the one or more controllers 20 generate a notification for the driver of the vehicle 12. As described above, the notification includes a visual message generated by the front display 22 notifying the driver that the physiological indicator of the occupant 14 has exceeded the minor event threshold. Figure 2C As shown, in one embodiment, the front display 22 also shows one or more cameras 28 ( Figure 1 ) real-time video transmission of the captured image data 64. The method 300 may then terminate.

[0065] In general, with reference to the accompanying drawings, the disclosed occupant monitoring system has various technical effects and benefits. Specifically, the occupant monitoring system provides a method for the driver to monitor one or more occupants in the vehicle without requiring the driver to turn his head or body away from the windshield of the vehicle. The occupant monitoring system also provides notifications to the driver when the occupant needs to take immediate action or when the occupant is experiencing a physical problem that only needs to be monitored by the vehicle driver. It should be understood that the occupant monitoring system may be particularly advantageous when young children or elderly people who cannot communicate verbally are sitting in the back seat of the vehicle because the occupant monitoring system does not require the rear occupants to communicate verbally or complete affirmative actions.

[0066] A controller may refer to an electronic circuit, a combinational logic circuit, a field programmable gate array (FPGA), a (shared, dedicated or group) processor that executes code, or a combination of some or all of the above, such as in a system on a chip. In addition, the controller may be microprocessor-based, for example, a computer having at least one processor, memory (RAM and / or ROM), and associated input and output buses. The processor may run under the control of an operating system residing in the memory. The operating system may manage computer resources so that computer program code embodied as one or more computer software applications (such as applications residing in the memory) may have instructions executed by the processor. In an alternative embodiment, the processor may execute the application directly, in which case the operating system may be omitted.

[0067] The description of the present disclosure is merely exemplary in nature, and variations that do not depart from the gist of the present disclosure are intended to fall within the scope of the present disclosure. These variations should not be regarded as departing from the spirit and scope of the present disclosure.

Claims

1. An occupant monitoring system for monitoring one or more occupants located on vehicle seats, the occupant monitoring system comprising: one or more cameras positioned to capture image data representative of the occupant; one or more biometric sensors, each of the one or more biometric sensors monitoring a parameter indicative of a physiological indicator of the occupant; as well as One or more controllers in electronic communication with the one or more cameras and the one or more biometric sensors, wherein the one or more controllers execute instructions to: determining one or more physiological indicators of the occupants based on the image data collected by the one or more cameras and one or more parameters indicative of one or more physiological indicators of each occupant collected by the one or more biometric sensors; comparing the occupant's physiological indicators with baseline values, wherein the baseline value of each physiological indicator indicates that the occupant is functioning normally; In response to determining that the physiological indicator falls outside of an allowed range of the baseline value, comparing the physiological indicator to at least one of a major event threshold and a minor event threshold; and In response to determining that the physiological indicator exceeds the major event threshold or the minor event threshold, a notification is generated to the vehicle driver.

2. The occupant monitoring system according to claim 1, wherein: The one or more controllers execute instructions to: determining that the physiological indicator exceeds the major event threshold; and In response to determining that the physiological indicator exceeds the major event threshold, it is determined that the occupant requires immediate action.

3. The occupant monitoring system according to claim 2, wherein: The notification includes a visual alarm and a voice alarm.

4. The occupant monitoring system according to claim 2, wherein: The one or more controllers execute instructions to: The vehicle is instructed to perform one or more of the following corrective actions: switch to a fully autonomous driving mode so that the driver can attend to the occupants, navigate the vehicle to the nearest hospital, contact emergency personnel via an onboard wireless communication system, and perform an automatic pull-over via an automated driving system.

5. The occupant monitoring system according to claim 1, wherein: The one or more controllers execute instructions to: determining that the physiological indicator exceeds the secondary event threshold; and In response to determining that the physiological indicator exceeds the minor event threshold, it is determined that the occupant is experiencing a physical problem that requires monitoring by the vehicle driver.

6. The occupant monitoring system according to claim 5, wherein: The notification includes a visual message notifying the driver that the physiological indicator has exceeded the minor event threshold.

7. The occupant monitoring system of claim 1 further comprising a front row display in electronic communication with the one or more controllers, wherein: The front display generates visual content that is visible to a driver of the vehicle.

8. The occupant monitoring system according to claim 7, wherein: The front display is one of the following: an augmented reality head-up display AR-HUD and an infotainment screen.

9. The occupant monitoring system according to claim 7, wherein: The one or more controllers execute instructions to: The front row display is directed to generate visual content representative of a physiological indicator of each of the one or more occupants.

10. The occupant monitoring system according to claim 9, wherein: The visual content includes a real-time video feed based on image data captured by the one or more cameras.

Images