OCCUPANT MONITORING SYSTEM FOR A VEHICLE

Abstract

An occupant monitoring system for monitoring one or more occupants located in a vehicle seat comprises one or more cameras positioned to capture image data representing an occupant, one or more biometric sensors each monitoring a parameter representing a physiological measurement of the occupant, and one or more controllers.The one or more controllers execute instructions to determine one or more physiological measurements of the occupant based on the image data collected by the one or more cameras and one or more parameters indicating the one or more physiological measurements of each occupant collected by the one or more biometric sensors, and in response to the determination that the physiological measurement exceeds either the threshold for a serious event or the threshold for a minor event, to generate a notification directed to the driver of the vehicle.

Description

INTRODUCTION

[0001] The present disclosure relates to an occupant monitoring system that monitors one or more physiological measurements of one or more occupants located in a seat of a vehicle.

[0002] Children who are twelve years old or younger, or who meet a minimum height requirement, should generally sit in a rear seat of a vehicle. In many situations, the caregiver for a child in the rear seat is the driver of the vehicle. Consequently, the caregiver may need to turn their head away from the roadway visible through the windshield to look at the child in the rear seat of the vehicle. Accordingly, it may be challenging for the caregiver to give the child attention or supervise the child while simultaneously driving the vehicle. For example, a very young child, such as an infant or toddler, may become upset, agitated, or have a tantrum while traveling in a vehicle. It may be difficult for the caregiver to calm the toddler and simultaneously drive the vehicle.Furthermore, it is obvious that very young children and other individuals, such as the elderly, may not be able to verbally communicate their health problems or discomfort. This can also cause problems, as the caregiver is unable to check on or care for the person while driving the vehicle.

[0003] Although current vehicles serve their intended purpose, there is a need in the art for an improved approach to monitoring rear seat occupants by a vehicle driver. SUMMARY

[0004] According to several aspects, an occupant monitoring system for monitoring one or more occupants seated in a vehicle is disclosed. The occupant monitoring system includes one or more cameras positioned to capture image data representative of an occupant, one or more biometric sensors each monitoring a parameter indicative of a physiological measurement of the occupant, and one or more controllers in electronic communication with the one or more cameras and the one or more biometric sensors.The one or more controllers execute instructions to determine one or more physiological measurements of an occupant based on the image data collected by the one or more cameras and one or more parameters indicative of the one or more physiological measurements of each occupant collected by the one or more biometric sensors. The one or more controllers compare a physiological measurement of the occupant to a baseline value, where the baseline value for each physiological measurement indicates that the occupant is in a normal condition. In response to determining that the physiological measurement is outside an acceptable range of the baseline value, the one or more controllers compare the physiological measurement to at least one major event threshold and a minor event threshold.In response to the determination that the physiological measurement exceeds either the threshold for a serious event or the threshold for a minor event, one or more controllers generate a notification directed to the driver of the vehicle.

[0005] In another aspect, the one or more controllers execute instructions to determine that the physiological reading exceeds the severe event threshold, and in response to determining that the physiological reading exceeds the severe event threshold, determine that the occupant requires immediate action.

[0006] In another aspect, the notification includes a visual and an audible warning.

[0007] In one aspect, the one or more controllers execute instructions to direct the vehicle to perform one or more of the following corrective actions: switching to a fully autonomous driving mode to allow the driver to care for the occupant, navigating the vehicle to the nearest hospital, contacting emergency personnel via an in-vehicle wireless communication system, and performing an automated roadside maneuver by an automated driving system.

[0008] In another aspect, the one or more controllers execute instructions to determine that the physiological measurement exceeds the minor event threshold, and in response to determining that the physiological measurement exceeds the minor event threshold, determine that the occupant has a physical issue requiring monitoring by the driver of the vehicle.

[0009] In another aspect, the notification includes a visual message informing the driver that the physiological measurement has exceeded the threshold for a minor event.

[0010] In another aspect, the occupant monitoring system further comprises a front row display in electronic communication with one or more controllers, the front row display generating visual content visible to a driver of the vehicle.

[0011] In one aspect, the front row display is one of the following: an augmented reality head-up display (AR-HUD) and an infotainment screen.

[0012] In another aspect, the one or more controllers execute instructions to direct the front row display to generate visual content representing the physiological measurements of each of the one or more occupants.

[0013] In another aspect, the visual content includes a live video feed based on image data captured by one or more cameras.

[0014] In one aspect, the occupant monitoring system further comprises one or more occupant databases in electronic communication with the one or more controllers that store facial recognition data representing facial characteristics of one or more known individuals and data indicative of baseline physiological metrics corresponding to a specific known individual.

[0015] In another aspect, the one or more controllers execute instructions to execute one or more facial recognition algorithms to determine an identity of the occupant based on the image data captured by the one or more cameras and compare the identity of the occupant to the facial recognition data of the one or more known individuals stored in the one or more occupant databases.

[0016] In another aspect, in response to the determination of a match between the identity of the occupant and the facial recognition data representing one of the known persons, the controllers use the baseline values ​​for the physiological measurements corresponding to the specific known person stored in the one or more occupant databases.

[0017] In one aspect, in response to determining that there is no match, the one or more controllers use default baseline values ​​for the physiological measurements.

[0018] In another aspect, the one or more cameras include one or more of the following: a red, green and blue camera (RGB camera) that captures visible light image data, an infrared camera that captures infrared image data, and a thermal imaging camera that captures thermal image data.

[0019] Another aspect of physiological measurements includes one or more of the following: heart rate, respiratory rate, body temperature, muscle stiffness, sweat rate, drowsiness, stress, motion sickness, pupil dilation, blood glucose level, and blood pressure.

[0020] In one aspect, the occupant monitoring system further comprises one or more radar sensors in electronic connection with one or more controllers, wherein the one or more radar sensors generate signals indicating the presence and seating position of the occupant.

[0021] In another aspect, a method for monitoring one or more occupants seated in a vehicle seat by an occupant monitoring system is disclosed. The method includes determining, by one or more controllers, one or more physiological measurements of an occupant based on image data collected by one or more cameras and one or more parameters indicative of one or more physiological measurements of each occupant collected by one or more biometric sensors. The method includes comparing, by the one or more controllers, a physiological measurement of the occupant to a baseline value, wherein the baseline value for each physiological measurement indicates that the occupant is in a normal condition.In response to determining that the physiological measurement is outside a permissible range of the baseline value, the method includes comparing the physiological measurement to a major event threshold and a minor event threshold. In response to determining that the physiological measurement exceeds either the major event threshold or the minor event threshold, the method includes generating a notification directed to the driver of the vehicle.

[0022] In another aspect, an occupant monitoring system for monitoring one or more occupants seated in a vehicle seat is disclosed. The occupant monitoring system includes one or more cameras positioned to capture image data representative of an occupant, one or more biometric sensors each monitoring a parameter indicative of a physiological reading of the one or more occupants, and one or more controllers in electronic communication with the one or more cameras and the one or more biometric sensors.The one or more controllers execute instructions to determine one or more physiological measurements of an occupant based on the image data collected by the one or more cameras and one or more parameters indicative of the one or more physiological measurements of each occupant collected by the one or more biometric sensors. The one or more controllers compare a physiological measurement of the occupant to a baseline value, where the baseline value for each physiological measurement indicates that the occupant is in a normal condition. In response to determining that the physiological measurement is outside an acceptable range of the baseline value, the one or more controllers compare the physiological measurement to a severe event threshold.In response to determining that the physiological measurement exceeds the threshold for a severe event, the one or more controllers generate a notification directed to the driver of the vehicle and determine that the occupant requires immediate action.

[0023] In another aspect, in response to determining that the physiological reading does not exceed the major event threshold, the one or more controllers compare the physiological reading to a minor event threshold, determine that the physiological reading exceeds the minor event threshold, and in response to determining that the physiological reading exceeds the minor event threshold, determine that the occupant has a physical issue that requires monitoring by the driver of the vehicle.

[0024] Further areas of applicability will become apparent from the description provided herein. It should be understood that the description and specific examples are for purposes of illustration only and are not intended to limit the scope of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings described herein are for illustrative purposes only and are not intended to limit the scope of the present disclosure in any way. Fig. 1 is a schematic illustration of a vehicle incorporating the disclosed occupant monitoring system for monitoring one or more physiological measurements of one or more occupants according to an exemplary embodiment; Fig. 2A is an exemplary illustration of an infotainment screen of the occupant monitoring system, the infotainment screen displaying visual content representing the physiological measurements of each occupant in the vehicle, according to an exemplary embodiment, Fig. 2B is an exemplary illustration of another design of the infotainment screen, which is found in Fig. 2A is shown, with the infotainment screen displaying the physiological measurements of each occupant based on a seating position according to an exemplary embodiment; Fig. 2C is an exemplary representation of another design of the infotainment screen, which is found in Fig. 2A is shown, with the infotainment screen displaying a live video feed of image data representing one or more occupants according to an exemplary embodiment; and Fig. 3 illustrates a process flow diagram illustrating a method for monitoring one or more of the Fig. 1 by the disclosed occupant monitoring system according to an exemplary embodiment. DETAILED DESCRIPTION

[0026] The following description is merely exemplary and is not intended to limit the present disclosure, application, or uses.

[0027] With reference to Fig. Figure 1 illustrates an occupant monitoring system 10 for a vehicle 12. As explained below, the occupant monitoring system 10 monitors one or more physiological measurements of one or more occupants 14 located in one of the seats 16 of the vehicle 12. In a non-restrictive configuration, the seat 16 is part of the second, third, or fourth row of seats in the vehicle 12. In one configuration, the one or more occupants 14 to be monitored are located within an interior space 18 of the vehicle 12 but are not seated in one of the seats in that interior space 18. In one configuration, the one or more occupants 14 include a person requiring special attention, such as an infant, a toddler, a child under 12 years of age, or an elderly person. In the alternative configuration, the one or more occupants 14 may also include a pet, such as a dog or a cat.It will be appreciated that the vehicle 12 may be any type of vehicle, such as, but not limited to, a sedan, a truck, a sport utility vehicle, a van, a bus, or a motor home.

[0028] The occupant monitoring system 10 includes one or more controllers 20 in electronic communication with a front-row display 22, a speaker 24, one or more radar sensors 26, one or more cameras 28, and one or more biometric sensors 30. In one embodiment, the one or more controllers 20 are in wireless communication with a personal electronic device 32 associated with the driver of the vehicle 12. The personal electronic device 32 is any type of portable electronic device with wireless computing capabilities that includes one or more processors and a display 36 that displays images and graphics to the driver, such as a smartphone or tablet computer.

[0029] The front row display 22 is positioned within the interior 18 of the vehicle 12 and generates visual content, such as graphics and text, that is visible to the driver of the vehicle 12. It will be appreciated that the visual content generated by the front row display 22 is visible when the driver is steering the vehicle 12 without the driver having to turn their head or body toward the rear of the vehicle 12. In one non-limiting embodiment, the front row display 22 is an augmented reality head-up display (AR-HUD) that generates the visual content directly onto a windshield of the vehicle 12 in the front field of view of the driver. In another embodiment, the front row display 22 alternatively includes an infotainment screen 34 ( Fig. 2A) that displays the visual content, wherein the infotainment screen is mounted on a dashboard 38 of the vehicle 12.

[0030] The one or more radar sensors 26 provide signals to the one or more controllers 20 indicating the presence and seating position of the occupant 14. The one or more cameras 28 are located within the interior 18 of the vehicle 12 and are positioned to capture image data representative of the occupant 14 seated in the seats 16. The one or more cameras 28 include red, green, and blue (RGB) cameras that capture visible light image data, infrared cameras that capture infrared image data, or thermal cameras that capture thermal image data. It will be appreciated that the visible light image data includes infrared image data, and thermal image data.

[0031] The one or more biometric sensors 30 each monitor a parameter indicative of a physiological measurement of the occupant 14 seated in the seat 16 of the vehicle 12. The physiological measurement represents a bodily function of a person. Some examples of physiological measurements of the occupant 14 include, among others, heart rate, respiratory rate, body temperature, muscle stiffness, sweat rate, drowsiness, stress, motion sickness, pupil dilation, blood glucose level, and blood pressure. It will be appreciated that in embodiments, the one or more biometric sensors 30 may include the one or more radar sensors 26 and the one or more cameras 28.Some examples of one or more biometric sensors 30 include, but are not limited to, a heart rate sensor, a galvanic skin response (GSR) sensor that detects electrical conductivity of the skin of the occupant 14, and a piezoelectric muscle sensor that detects a piezoelectric resonance of the occupant 14, where the piezoelectric resonance is indicative of muscle stiffness. In embodiments, the GSR sensor is mounted on a surface of the door handles or an armrest to contact the skin of an occupant 14.

[0032] In embodiments, the one or more controllers 20 determine the sweat rate of the occupant 14 based on thermal image data representing a face of the occupant 14, and in particular a forehead of the occupant 14, acquired by the one or more thermal imaging cameras 28. In embodiments, the sweat rate of the occupant 14 is further determined based on visible light image data also acquired by one or more RGB cameras 28.

[0033] In this configuration, the occupant's drowsiness 14 is determined based on image data captured by one or more cameras 28, the occupant's heart rate recorded by the heart rate sensor, and the electrical conductivity of the occupant's skin measured by the GSR sensor. The image data captured by the one or more cameras 28, for example, show various behaviors of the occupant 14 that indicate drowsiness, such as yawning, closing the eyes, blinking, and head posture. Furthermore, it is evident that the occupant's heart rate 14 varies considerably between the different stages of drowsiness, such as alertness and fatigue.

[0034] In one embodiment, the one or more controllers 20 determine the stress level of the occupant 14 based on the occupant's heart rate as measured by the heart rate sensor and the electrical conductivity of the occupant's skin as measured by the GSR sensor. In another embodiment, the one or more controllers 20 also determine the occupant's stress level based on image data collected by the one or more cameras 28. In one embodiment, the occupant's motion sickness 14 is determined based on the pupillary rhythms monitored by image data acquired by one or more cameras 28.

[0035] Further referring to Fig. 1, during initialization of the occupant monitoring system 10, the one or more controllers 20 monitor the image data representing the occupant 14 positioned in the seat 16 and captured by the one or more cameras 28. The one or more controllers 20 execute one or more facial recognition algorithms to determine the identity of the occupant 14 based on the image data representing the occupant 14. The one or more controllers 20 are in electronic communication with one or more occupant databases 44 that store facial recognition data representing the facial features of one or more known individuals. The one or more occupant databases 44 also store data indicative of baseline physiological metrics corresponding to a specific known individual.In some configurations, the inmate databases 44 are stored locally, but the inmate databases 44 can also be stored on a network server.

[0036] The one or more controllers 20 execute instructions to compare the identity of the occupant 14 with the facial recognition data stored in the one or more occupant databases 44. In response to identifying a match between the identity of the occupant 14 and one of the known individuals stored in the occupant database 44, the one or more controllers 20 use the baseline physiological readings corresponding to the specific known individual stored in the one or more occupant databases 44. However, if no match is identified, the one or more controllers 20 use default baseline physiological readings of the occupant 14.

[0037] The one or more controllers 20 execute instructions to determine one or more physiological measurements of each occupant 14 based on one or more of the following: the signals received from the one or more radar sensors 26, the image data collected by the one or more cameras 28, and the one or more parameters indicative of the one or more physiological measurements of each occupant 14 collected by the one or more biometric sensors 30. The one or more controllers 20 instruct the front-row display 22 to generate visual content representing the physiological measurements of each occupant 14. Fig. For example, Figure 2A is an exemplary illustration of the front row display 22, wherein the front row display 22 includes the infotainment screen 34 that displays visual content representing the physiological measurements of each occupant 14 located in the vehicle 12. In the Fig. 2A, the infotainment screen 34 displays an icon 50 corresponding to one of the occupants 14 and one or more icons 52 representing physiological measurements for each occupant 14 located in the interior 18 of the vehicle 12.

[0038] In the Fig. 2A, the icon 50 represents the gender of the corresponding occupant 14, and the one or more icons 52 represent a heart rate and a respiratory rate corresponding to each occupant 14 in the vehicle 12. However, it will be appreciated that the driver or other occupant may customize the display in the front row 22 so that the icons 52 instead represent different or additional physiological measurements.

[0039] Fig. 2B is an alternative configuration of the infotainment screen 34, which is in Fig. 2A is shown, with the infotainment screen 34 displaying the physiological measurements of each of the occupants 14 based on a seating position. In the Fig. 2B, a position on the rear passenger seat 60 and a position on the rear driver seat 62 are shown, with each seat position 60, 62 being associated with one or more symbols 52 representing physiological measurements corresponding to a respective occupant 14.

[0040] Fig. 2C is a further embodiment of the infotainment screen 34, which is shown in Fig. 2A, wherein the infotainment screen 34 displays a live video feed 64 based on the image data captured by one or more cameras 28 ( Fig. 1) of the occupants 14. The infotainment screen 34 also displays the icon 50 corresponding to a selected one of the occupants 14 and one or more icons 52 representing physiological measurements for the selected occupant 14. It will be appreciated that the driver may switch the view of the infotainment screen 34 to display the physiological measurements of one of the remaining occupants 14, if applicable, using a touchscreen function, a button, or based on an automatic switching function.

[0041] With reference to Fig. 1, the one or more controllers 20 execute instructions to continuously compare each physiological measurement of the occupant 14 to a baseline value. The baseline value corresponds to a specific physiological measurement and indicates that the occupant 14 is in a normal condition and no health-related event is present. In other words, the baseline value corresponding to a specific physiological measurement indicates that the occupant 14 does not have a physiological problem that requires immediate action or necessitates driver monitoring. Some examples of physiological problems that necessitate driver monitoring include an elevated heart rate, elevated body temperature, and high stress. It is apparent that physiological problems that necessitate driver monitoring do not require actual intervention.In contrast, the occupant 14 may sometimes experience a physiological problem that requires driver intervention, such as a heart attack, an anxiety attack, or an asthma attack. In embodiments, the baseline value indicates an acceptable range of values ​​for a corresponding physiological measurement, indicating that a person is in a normal state.

[0042] In response to determining that a physiological reading is outside the acceptable range of baseline values, the one or more controllers 20 compare the physiological reading to a severe event threshold. The one or more controllers 20 also compare the physiological reading to a minor event threshold. The severe event threshold indicates that the occupant 14 is experiencing a physiological issue that requires immediate action by the driver of the vehicle 12. In response to determining that the physiological reading exceeds the severe event threshold, the one or more controllers 20 determine that the occupant 14 requires immediate action.Some examples of physiological measurements represented by the threshold for a severe event include, but are not limited to, high-grade fever (over approximately 39.5°C or 103°F) or a relatively severe drop in blood pressure (at least 20 millimeters of mercury (mmHg)).

[0043] In response to determining that the occupant 14 requires immediate action, the one or more controllers 20 generate a notification directed to the driver of the vehicle 12. In one embodiment, the notification includes a visual alert generated by the front-row display 22 and an audible alert generated by the speaker 24. The audible alert may include, for example, an emergency graphic and a message indicating that the occupant 14 requires immediate action, and the audible alert may include an alarm, a tone, or a voice message indicating that immediate action is required. In one embodiment, the one or more controllers 20 may transmit a signal instructing the personal electronic device 32 associated with the driver of the vehicle 12 to also generate the visual alert, the audible alert, or both.It will be appreciated that the personal electronic device 32 may also be used in situations where the driver has temporarily moved away from the vehicle 12.

[0044] In embodiments, in response to determining that the occupant 14 requires immediate attention, the one or more controllers 20 instruct the vehicle 12 to perform one or more of the following corrective actions: switching to a fully autonomous driving mode to allow the driver to care for the occupant 14, navigating the vehicle 12 to the nearest hospital or emergency room, contacting emergency personnel via an in-vehicle wireless communication system, and performing an automated roadside maneuver by an automated driving system. In one embodiment, the one or more controllers 20 may instruct the front-row display 22 to generate a graphic prompting the driver of the vehicle 12 to select one of the corrective actions.

[0045] The minor event threshold indicates that the occupant 14 is experiencing a physiological issue that requires monitoring by the driver of the vehicle 12, but does not require immediate action. In response to determining that the physiological reading exceeds the minor event threshold, the one or more controllers 20 generate a notification. If the occupant 14 does not require immediate attention, in one embodiment, the notification includes a visual message generated by the front-row display 22 informing the driver of the physiological reading of the occupant 14. As in Fig. 2C, in one embodiment, the display in the front row 22 generates a symbol 50 corresponding to the occupant 14 and a symbol 52 representing the physiological measurements. In one embodiment, the display in the front row 22 also shows the live video feed 64 of the information captured by one or more cameras 28 ( Fig. 1) captured image data representing occupant 14. Some examples of physiological measurements represented by the minor event threshold include, but are not limited to, low-grade fever (between 37.7 and 39.5°C or 100-103°F).

[0046] In one embodiment, the one or more controllers 20 execute one or more object detection and tracking algorithms to identify an object of interest within the control of the occupant 14, track the object of interest, and determine when the occupant 14 has lost control of the object of interest based on the image data captured by the one or more cameras 28. For example, in one embodiment, the occupant 14 is a toddler, and the object of interest is a favorite toy or pacifier. The one or more controllers 20 analyze the image data captured by the one or more cameras 28 to identify the object of interest located within one or more hands or lap of the occupant 14 based on one or more object detection algorithms.The one or more controllers 20 execute tracking algorithms to continuously track the subject object. The one or more controllers 20 determine that the occupant 14 has lost control of the subject object in response to detecting that the subject object has fallen to the floor of the interior 18 of the vehicle 12. The one or more controllers 20 may also perform vomit detection, monitoring the condition of the rear seat belts, and determining whether the occupant 14 is sticking a body part out a window of the vehicle 12 based on one or more object detection and tracking algorithms.

[0047] In another embodiment, the one or more controllers 20 monitor the signals received from the one or more radar sensors 26, the image data collected by the one or more cameras 28, and the one or more parameters indicative of the one or more physiological measurements of each occupant 14 collected by the one or more biometric sensors 30 to determine a comfort level of the occupant 14. The comfort level of the occupant 14 is determined based on the breathing rate and sweat rate of the occupant 14. It will be appreciated that in some cases, the driver temporarily moves away from the vehicle 12, but the occupant 14 remains in the seat 16.The one or more controllers 20 compare the comfort level to a respiratory rate threshold and a sweat rate threshold, where the respiratory rate threshold and the sweat rate threshold indicate occupant discomfort. In embodiments, in response to determining the comfort level exceeding either the respiratory rate threshold or the sweat rate threshold, the one or more controllers 20 instruct a heating, ventilation, and air conditioning (HVAC) system within the cabin to adjust the temperature within the cabin 18. For example, during warmer temperatures, the cabin air conditioning system increases cooling for the occupant 14, and during cooler temperatures, the cabin air conditioning system increases heating for the occupant 14.In one embodiment, the one or more controllers 20 may transmit a signal instructing the electronic device 32 associated with the driver of the vehicle 12 to also generate the visual warning, the audible warning, or both to indicate that the occupant 14 is experiencing discomfort.

[0048] Fig. 3 is a process flow diagram illustrating a method 300 for monitoring the one or more occupants 14 located on the seat 16 of a vehicle 12 by the occupant monitoring system 10. Referring to the two Fig. 1 and Fig. 3, the method 300 may begin with block 302. In block 302, the one or more controllers 20 monitor the image data captured by the one or more cameras 28 depicting the occupant 14 positioned in the seat 16 of the vehicle 12. The method 300 may then proceed to block 302.

[0049] At block 302, the one or more controllers 20 execute one or more facial recognition algorithms to determine the identity of the occupant 14 based on the image data. The method 300 may then proceed to decision block 304.

[0050] At decision block 306, the one or more controllers 20 compare the identity of the occupant 14 with the facial recognition data of the one or more known individuals stored in the one or more occupant databases 44 to determine if there is a match between the identity of the occupant 14 and the facial recognition data representing one of the known individuals. In response to determining that there is a match, the method continues to block 308. At block 308, the one or more controllers 20 use the baseline physiological measurements corresponding to the specific known individual stored in the one or more occupant databases 44, and the method 300 continues to block 312.

[0051] In response to determining that there is no match, the method proceeds to block 310. At block 310, the one or more controllers 20 use the default baseline values ​​for the physiological measurements. The method 300 may then proceed to block 312.

[0052] At block 312, the one or more controllers 20 execute instructions to determine one or more physiological measurements of the occupant 14 based on one or more of the following: the signals received from the one or more radar sensors 26, the image data collected by the one or more cameras 28, and the one or more parameters indicative of the one or more physiological measurements of each occupant 14 collected by the one or more biometric sensors 30. The method 300 may then proceed to block 314.

[0053] In block 314, the one or more controllers 20 instruct the display in the front row 22 to generate visual content representing the physiological measurements of each occupant 14, as in Fig. 2A-2C. The method 300 may then proceed to decision block 316.

[0054] At decision block 316, the one or more controllers 20 compare a physiological measurement of the occupant 14 to a baseline value, where the baseline value for the physiological measurement indicates that the occupant is in a normal condition. In response to determining that the physiological measurement is outside the acceptable range of the baseline value, the method 300 proceeds to decision block 318. Otherwise, the method 300 returns to block 312.

[0055] At decision block 318, the one or more controllers 20 compare the physiological reading to the severe event threshold. In response to determining that the physiological reading exceeds the severe event threshold, the one or more controllers 20 determine that the occupant 14 requires immediate action, and the method continues to block 320. Otherwise, the method 300 may continue to decision block 324.

[0056] At block 320, in response to determining that the occupant 14 requires immediate action, the one or more controllers 20 generate the notification directed to the driver of the vehicle 12. As mentioned above, in one embodiment, the notification includes a visual alert generated by the front-row display 22 and an audible alert generated by the speaker 24. The method 300 may then proceed to block 322.

[0057] At block 322, the one or more controllers 20 instruct the vehicle 12 to perform one or more of the following corrective actions: switching to a fully autonomous driving mode so the driver can care for the occupant 14, navigating the vehicle 12 to the nearest hospital, contacting emergency personnel via an in-vehicle wireless communication system, and performing an automated roadside maneuver by an automated driving system. As noted above, in embodiments, the one or more controllers 20 may instruct the front-row display 22 to generate a graphic prompting the driver of the vehicle 12 to select one of the corrective actions. The method 300 may then terminate or return to block 312.

[0058] At decision block 324, the one or more controllers 20 compare the physiological measurement to the minor event threshold. In response to determining that the physiological measurement exceeds the minor event threshold, the one or more controllers 20 determine that the occupant 14 has a physical issue requiring monitoring by the driver of the vehicle 12, and the method 300 proceeds to block 326. Otherwise, the method 300 returns to block 312.

[0059] In block 326, the one or more controllers 20 generate the notification directed to the driver of the vehicle 12. As mentioned above, the notification includes a visual message generated by the front-row display 22 informing the driver of the physiological reading of the occupant 14 that has exceeded the threshold for a minor event. As described in Fig. 2C, in one embodiment the display in the front row 22 also shows the live video feed 64 of the one or more cameras 28 ( Fig. 1) captured image data representing the occupant 14. The method 300 can then be terminated.

[0060] With general reference to the figures, the disclosed occupant monitoring system provides various technical effects and advantages. In particular, the occupant monitoring system provides the driver with an approach for monitoring one or more occupants within the vehicle without requiring the driver to turn their head or body away from the vehicle's windshield. The occupant monitoring system also provides notifications to the driver when the occupant requires immediate attention or when the occupant merely has a physical problem that requires monitoring by the vehicle's driver.It is obvious that the occupant monitoring system can be particularly beneficial when a very young child or an elderly person who is unable to communicate verbally is in the rear seat of the vehicle, since the occupant monitoring system does not require any verbal communication or affirmative action from the rear occupant.

[0061] The controllers may refer to or be part of an electronic circuit, a combined logic circuit, a field-programmable gate array (FPGA), a (shared, dedicated, or group) processor executing code, or a combination of some or all of the above, such as in a system on a chip. In addition, the controllers may be microprocessor-based, such as a computer having at least one processor, memory (RAM and / or ROM), and associated input and output buses. The processor may operate under the control of an operating system residing in memory. The operating system may manage computer resources so that computer program code embodied as one or more computer software applications, such as an application residing in memory, can have instructions executed by the processor.In an alternative embodiment, the processor can execute the application directly, in which case the operating system can be omitted.

[0062] The description of the present disclosure is merely exemplary, and variations that do not depart from the gist of the disclosure are to be considered within the scope of the present disclosure. Such variations should not be construed as a departure from the spirit and scope of the present disclosure. legend

[0063] In the drawings, N stands for No and Y stands for Yes.

Claims

[1] Occupant monitoring system for monitoring one or more occupants located on a seat of a vehicle, the occupant monitoring system comprising one or more cameras positioned to capture image data representing an occupant; one or more biometric sensors, each monitoring a parameter indicating a physiological measurement of the occupant; and one or more controllers in electronic communication with the one or more cameras and the one or more biometric sensors, the one or more controllers executing instructions to: Determining one or more physiological measurements of an occupant based on the image data collected by the one or more cameras and one or more parameters indicative of the one or more physiological measurements of each occupant collected by the one or more biometric sensors; Comparing a physiological measurement of the occupant with a baseline value, wherein the baseline value for each physiological measurement indicates that the occupant is in a normal condition; in response to determining that the physiological measurement is outside an acceptable range of the baseline value, comparing the physiological measurement to at least one of a major event threshold and a minor event threshold; and in response to determining that the physiological measurement exceeds either the major event threshold or the minor event threshold, generating a notification directed to the driver of the vehicle. [2] The occupant monitoring system of claim 1, wherein the one or more controllers execute instructions to: Determining that the physiological measurement exceeds the threshold for a serious event; and in response to determining that the physiological measurement exceeds the threshold for a serious event, determining that the occupant requires immediate action. [3] The occupant monitoring system of claim 2, wherein the notification comprises a visual warning and an audible warning. [4] The occupant monitoring system of claim 2, wherein the one or more controllers execute instructions to: Instruct the vehicle to perform one or more of the following corrective actions: switching to a fully autonomous driving mode so the driver can care for the occupant, navigating the vehicle to the nearest hospital, contacting emergency personnel via an in-vehicle wireless communication system, and performing an automated roadside maneuver through an automated driving system. [5] The occupant monitoring system of claim 1, wherein the one or more controllers execute instructions to: Determining that the physiological measurement exceeds the threshold for a minor event; and in response to determining that the physiological measurement exceeds the threshold for a minor event, determining that the occupant is experiencing a physical condition that requires monitoring by the driver of the vehicle. [6] The occupant monitoring system of claim 5, wherein the notification comprises a visual message informing the driver that the physiological measurement has exceeded the minor event threshold. [7] The occupant monitoring system of claim 1, further comprising a front row display in electronic communication with the one or more controllers, the front row display generating visual content visible to a driver of the vehicle. [8] The occupant monitoring system of claim 7, wherein the front row display is one of the following: an augmented reality head-up display (AR-HUD) and an infotainment screen. [9] The occupant monitoring system of claim 7, wherein the one or more controllers execute instructions to: Instructing the front row display to generate visual content depicting the physiological measurements of each of the one or more occupants. [10] The occupant monitoring system of claim 9, wherein the visual content comprises a live video feed based on the image data captured by one or more cameras.

Images