Method and system for recommending rest-stops to a vehicle occupant

Abstract

A method and system for recommending rest-stops to a vehicle occupant (202) are disclosed. The method (100) includes obtaining (102) one or more first level characteristics of the vehicle occupant (202); processing (104), by a first trained neural network, the one or more first level characteristics to obtain a cumulative score of the vehicle occupant (202); characterized in that: obtaining 10 (106) one or more second level characteristics of the vehicle occupant (202); processing (108), by a second trained neural network, the one or more second level characteristics to determine a comfort score of the vehicle occupant (202); and presenting (110) a plurality of rest-stops to the vehicle occupant (202) based on the cumulative score and the comfort score.

Description

Field of Invention

[001] This invention generally relates to methods and systems for recommending rest-stops to a vehicle occupant. Background of Invention

[002] Vehicles assist to transport people from one destination to another. In some of such journeys, the distance between destinations or multiple destinations can be vast and it would be helpful for the occupants in a vehicle, or vehicle occupants, to occasionally be provided suggestions on rest stops during such a lengthy journey.

[003] A typical method for recommending rest-stops may include suggesting suitable programs based on previous history and actions recorded for in-cabin occupant refreshment. However, current methods do not contain solutions for recommendation of rest-stops during travel and also do not propose alternative reststops if the vehicle occupant does not like recommendations or refreshment options.

[004] Current methods and systems also do not evaluate the vehicle occupant’s comfort level, which includes both the driver and the passenger, during travel and also do not suggest suitable places to take a rest accordingly. They also do not disclose collective approaches of personalization, learning and data driven model development for a rest-stop recommendation.

[005] The level of comfort of the passengers and / or the driver, collectively referred as vehicle occupant(s), must be considered before appropriate rest-stops are recommended. Further, an alternative suggestion for a rest-stop should be given to the vehicle occupant if they do not like the suggestions or refreshment possibilities. There is thus a need for a system and method that can offer accurate predictions at a lower computing cost while recommending appropriate rest-stops for the vehicle occupant.

[006] There is therefore a need to provide a method and system that overcome or at least ameliorate one or more of the disadvantages discussed above. Summary

[007] It is an object to provide a method and system that address one or more of the problems discussed above.

[008] According to a first aspect of the present invention, there is provided a method for recommending rest-stops to a vehicle occupant, comprising: obtaining one or more first level characteristics of the vehicle occupant; processing, by a first trained neural network, the one or more first level characteristics to obtain a cumulative score of the vehicle occupant; characterized in that: obtaining one or more second level characteristics of the vehicle occupant; processing, by a second trained neural network, the one or more second level characteristics to determine a comfort score of the vehicle occupant; and presenting a plurality of rest-stops to the vehicle occupant based on the cumulative score and the comfort score.

[009] Advantageously, recommending rest-stops for vehicle occupants can aid the subject vehicle occupant to take a rest and help the vehicle occupants to experience a comfortable travel experience. The method as disclosed can evaluate comfort level of the vehicle occupant and recommends the most suitable time and location to take a rest using the vehicle occupant’s behaviour and / or terrain information.

[0010] In an embodiment, the step of processing the one or more first level characteristics comprises creating a unique profile for the vehicle occupant; and assigning a weightage score for each of the one or more first level characteristics based on an importance of the first level characteristics to the vehicle occupant; wherein the cumulative score is obtained based on the sum of each of the weightage scores. [0011 ] Advantageously, assigning a weightage score to the first level characteristics can help to sieve out events that have the potential to impact on the accuracy of the disclosed method and system, thus improving its durability.

[0012] In an embodiment, the first level characteristics comprises at least one of: age, gender, identity, emotion, activity, pose, mental fatigue, distraction, drowsiness, thermal comfort and trip comfort.

[0013] Advantageously, monitoring the first level characteristics such as distraction, mental fatigue and health can help in understanding the vehicle occupants’ comfort level when the vehicle occupant(s) are not expressive.

[0014] In an embodiment, the step of obtaining the first level characteristics comprises capturing audio, video, physiological data, vehicle data and / or route information.

[0015] Advantageously, the method and system of the present invention may use the in-cabin camera to capture images and data which can increase the ease of use, cost-effectiveness, and adaptability.

[0016] In an embodiment, the step of obtaining one or more second level characteristics comprises determining, by the first trained neural network, if the cumulative score is above a predetermined threshold value.

[0017] Advantageously, by determining if the cumulative score is above a predetermined threshold value, the method and system of the present invention does not have to always compute one or more second level characteristics and thus can save on any unnecessary computational cost.

[0018] In an embodiment, the second level characteristics comprises at least one of: human interaction, in-cabin comfort, generic emotion and cognitive load.

[0019] Advantageously, by using a holistic approach of understanding human behaviour through one or more attributes such as facial expressions, body pose, sound analysis, visual and / or verbal interactions between humans, cognitive load, physiological data as well as vehicle data, the method and system of the present invention can evaluate the comfort level with credibility and may improve quality standards.

[0020] In an embodiment, the step of processing the one or more second level characteristics to determine a comfort score comprises analyzing, by the second trained neural network, the second level characteristics and historical data of the vehicle occupant.

[0021] Advantageously, by computing a wide array of features, the method and system of the present invention can enable accurate decision making for corrective measures to elevate occupant comfort.

[0022] In an embodiment, historical data comprises history of previous trips and current trip information of the vehicle occupant.

[0023] Advantageously, the method and system of the present invention can recommend suitable rest-stops considering the age and gender of the vehicle occupant, especially for vehicle occupants who need to rest due during the journey such as the elderly, infants and children. This can result in more accurate recommendations. For example, it can be important for suggesting rest-stops when a family is travelling and one of the senior members of the group feels uneasy at any point in the journey.

[0024] In an embodiment, the step of presenting the plurality of rest stops comprises ranking the plurality of rest stops based on a comfort level of the vehicle occupant, the comfort level comprising criticality level, distance and location of the rest-stop.

[0025] Advantageously, evaluation of the vehicle occupant’s comfort can be useful in many other industries such as the vehicle fleet management, Healthcare and Automotive sectors.

[0026] In an embodiment, the method further includes storing a preference of the vehicle occupant for neural network learning if the vehicle occupant selects a rest-stop different from the presented plurality of rest-stops; and presenting a second set of rest-stops to the vehicle occupant.

[0027] Advantageously, the method and system of the present invention may learn an individual’s preferences and can provide better personalised and optimised reststop suggestions.

[0028] According to a second aspect of the present invention, there is provided a system for recommending rest-stops to a vehicle occupant, comprising: one or more data capturing modules configured to obtain one or more first level characteristics and one or more second level characteristics of the vehicle occupant; a first neural network module configured to implement a first trained neural network for processing the one or more first level characteristics to obtain a cumulative score of the vehicle occupant; a second neural network module configured to implement a second trained neural network for processing the one or more second level characteristics to determine a comfort score of the vehicle occupant; and a recommendation module configured to present a plurality of rest-stops to the vehicle occupant based on the cumulative score and the comfort score.

[0029] Advantageously, the method and system of the present invention can provide ride-sharing companies an avenue to evaluate the comfort as well as suggest relevant rest-stops for vehicle occupant(s) and thus resulting in high user satisfaction.

[0030] In an embodiment, the first neural network module is further configured to: create a unique profile for the vehicle occupant; and assign a weightage score for each of the one or more first level characteristics based on an importance of the first level characteristics to the vehicle occupant; and determine if the cumulative score is above a predetermined threshold value, wherein the cumulative score is obtained based on the sum of each of the weightage scores.

[0031] In an embodiment, the one or more data capturing modules is further configured to capture audio, video, physiological data, vehicle data and / or route information.

[0032] In an embodiment, the second neural network module is further configured to analyze the second level characteristics and historical data of the vehicle occupant.

[0033] In an embodiment, the recommendation module is further configured to: rank the plurality of rest stops based on a comfort level of the vehicle occupant, the comfort level comprising criticality level, distance and location of the reststop; store a preference of the vehicle occupant for neural network learning if the vehicle occupant selects a rest-stop different from the presented plurality of rest-stops; and present a second set of rest-stops to the vehicle occupant. Brief Description of Drawings

[0034] Embodiments will be better understood and readily apparent to one of ordinary skill in the art from the following written description, by way of example only, and in conjunction with the drawings, in which:

[0035] Figure 1 shows a flow chart illustrating a method for recommending reststops to a vehicle occupant in accordance with an embodiment of the invention.

[0036] Figure 2 shows a schematic diagram illustrating the flow of information in a system for recommending rest-stops to a vehicle occupant according to an embodiment of the invention.

[0037] Figure 3 shows a detailed flow chart illustrating an implementation of the method according to the example embodiments.

[0038] Like numerals denote like parts. Detailed Description

[0039] Figure 1 shows a flow chart illustrating a method 100 for recommending rest-stops to a vehicle occupant in accordance with an embodiment of the invention. At step 102, the method includes obtaining one or more first level characteristics of the vehicle occupant. At step 104, the method includes processing, by a first trained neural network, the one or more first level characteristics to obtain a cumulative score of the vehicle occupant. At step 106, the method includes obtaining one or more second level characteristics of the vehicle occupant. At step 108, the method includes processing, by a second trained neural network, the one or more second level characteristics to determine a comfort score of the vehicle occupant. At step 110, the method includes presenting a plurality of reststops to the vehicle occupant based on the cumulative score and the comfort score. It can be appreciated that throughout the present specification, discussions utilizing the term “vehicle occupant” can refer to the driver as well as to any of the passengers that are present in the vehicle.

[0040] In an example embodiment, a family is going on a trip and an elderly person among them is feeling uncomfortable anytime during the trip. The Artificial lntelligence(AI)-based invention as currently disclosed herein can calculate the comfort for the elderly and recommend suitable rest-stops. The rest-stop recommendations may be based on the elderly person’s preferences along with the distance and location of the rest-stop while considering the criticality of the rest-stop. Embodiments of the present invention may also use several factors such as age and gender, preferred rest-stop location and the vehicle occupant’s feedback to personalize the experience according to the vehicle occupant’s comfort.

[0041] In an embodiment of the present invention, the system may keep improving itself by receiving the vehicle occupant’s feedback and rest-stop choices so as to suggest better recommendations for future trips. The system as disclosed can also recommend suitable rest-stop location for vehicle occupants based on their comfort score during trips. The present system may also use cameras, microphones, physiological sensors measuring heart and respiration rate as well as vehicle and environmental data such as weather, time, infotainment, HVAC (heating, ventilation and air-conditioning), trip data, route information etc. in order to determine the comfort level of the vehicle occupant and recommend suitable rest-stops.

[0042] Figure 2 shows a schematic diagram illustrating the flow of information in a system 200 for recommending rest-stops to a vehicle occupant 202 according to an embodiment of the invention. The system 200 includes one or more data capturing modules 204, a first neural network module 206, a second neural network module 208, a recommendation module 210, a profile database 212 and an event log database 214. The one or more data capturing modules 204 is in communication with the first neural network module 206 and the event log database 214 while the first neural network module 206 is in communication with the second neural network module 208 and the profile database 212. The second neural network module 208 is in communication with the recommendation module 210, the profile database 212 and the event log database 214 while the recommendation module 210 is in communication with the profile database 212 and the event log database 214.

[0043] At step A in Figure 2, the one or more data capturing modules 204 is configured to obtain one or more first level characteristics of the vehicle occupant 202. The one or more first level characteristics can be obtained via the one or more data capturing modules 204 by capturing audio data, video data, physiological data, vehicle data and / or route information of the journey. The captured audio data, video data and physiological data may include data related to the vehicle occupant 202.

[0044] At step B, the captured data is sent to the first neural network module 206. The captured data may also be stored in the event log database 214 at step C for use at a later stage by the second neural network module 208 or the recommendation module 210. The first neural network module 206 is configured to implement a first trained neural network for processing one or more first level characteristics to obtain a cumulative score of the vehicle occupant 202.

[0045] The first level characteristics of the vehicle occupant 202 may include at least one of: age, gender, identity, emotion, activity, pose, mental fatigue, distraction, drowsiness, thermal comfort and trip comfort. The first trained neural network can be an Artificial Intelligence (Al)-based neural network model such as support vector machines, decision trees, ensemble models, k-nearest neighbours models or Bayesian networks. It can be appreciated that the first trained neural network can be a multi-modal neural network and may contain other types of models including linear models and / or non-linear models as well as feed-forward neural networks, convolutional neural networks (CNN) or recurrent neural networks (RNN).

[0046] In an implementation, the age and gender of each vehicle occupant 202 can be obtained from images captured from the in-cabin camera. Using the captured images, the first trained neural network, which may be trained by images containing clearly visible faces of the vehicle occupants, classifies the vehicle occupant 202 based on age and gender. The first trained neural network can be a CNN-based architecture network, such as ResNet, or lnceptionV3, and may also be trained on real or simulated data that is collected.

[0047] In an implementation, the identity of each vehicle occupant 202 can be obtained from images captured from the in-cabin camera. Using the captured images, the first trained neural network, which may be trained by images containing clearly visible faces of the vehicle occupants, identifies each vehicle occupant 202. The first trained neural network in this case can be a CNN-based architecture network, such as ResNet, or lnceptionV3, and may be trained on real or simulated data that is collected. After each of the vehicle occupants 202 is identified, the first trained neural network creates a unique profile for each vehicle occupant 202 for the entire trip using a unique identifier to identify the particular vehicle occupant 202. The unique profile of each vehicle occupant, together with the unique identifier, can then be stored in the profile database 212, as shown in step D of Figure 2. The unique profile for each vehicle occupant 202 can be used repeatedly, for example by the second neural network module 208, to map the vehicle occupant’s behaviour whenever an event occurs.

[0048] In an implementation, emotion recognition of each vehicle occupant 202 can be obtained from images captured from the in-cabin camera, voice volume and tone of the vehicle occupant 202 from a microphone situated in the vehicle. Using the captured data, the first trained neural network, which may be trained by images containing clearly visible facial emotions, may then recognize basic emotional expressions of each vehicle occupant 202 such as anger, disgust, fear, happiness, sadness, and surprise. The first trained neural network can subsequently assign ratings for each recognized emotion, ranging from a minimum of 0 for having little or no emotion to a maximum of +5 for expressing high emotion. The first trained neural network in this case can be a CNN-based architecture network such as VGG16, ResNet, or lnceptionV3 and may be trained on real or simulated in-cabin data that is collected.

[0049] In an implementation, body-pose estimation of each vehicle occupant 202 can be obtained from images captured from the in-cabin camera. Using the captured images, the first trained neural network, which may be trained by images containing human bodies appearing in varying poses and orientations and identify and locate body parts of humans, identifies the size, location, and orientation of body parts for each vehicle occupant 202. This can be achieved by looking at a combination of the pose and the orientation of the subject vehicle occupant 202 and then estimates his or her body-pose. The first trained neural network in this case can be a CNN-based architecture network such as High-Resolution Net (HRNet), DeepPose, OpenPose, etc. and may be trained on real or simulated data that is collected.

[0050] In an implementation, the activity of each vehicle occupant 202 can be recognized from images captured from the in-cabin camera. Using the captured images, the first trained neural network, which may be trained by images containing humans performing general activities such as operating a phone, drinking coffee, etc., recognizes activities of the vehicle occupant 202 from a series of observations on the actions of the vehicle occupant 202 and the surrounding environmental conditions. The first trained neural network may then label the human activity of each vehicle occupant 202. The first trained neural network in this case can be a RNN-based architecture such as LSTM, DeepConvLSTM, etc. and may be trained on real or simulated data that is collected.

[0051] In an implementation, mental fatigue estimation of the vehicle occupant 202 can be obtained from data captured from the in-cabin camera, microphones and physiological sensors such as heart and respiration rate, in the vehicle. Using the captured data, the first trained neural network, which may be trained by images containing human faces, audio samples, numerical data from physiological sensors such as heart rate, respiratory rate, temperature, etc., estimates the temporary inability of the vehicle occupant 202 to maintain optimal cognitive performance. The first trained neural network may then determine a numerical measure of the mental fatigue of the vehicle occupant 202. The first trained neural network in this case may be trained on real or simulated data that is collected and can be a multi-modal neural network, including a CNN-based architecture such as EmCNN for analyzing images, a RNN-based architecture such as LSTM for analyzing data from physiological sensors and a RNN or a time-frequency convolutional neural network (TFCNN) for analyzing audio data.

[0052] In an implementation, distraction detection of the vehicle occupant 202 can be obtained from data captured from in-cabin camera and microphones in the vehicle. Using the captured data, the first trained neural network, which may be trained by images containing human body and audio signals, detects the vehicle occupant’s inattention and distraction and determines whether the vehicle occupant 202 is distracted. The first trained neural network in this case may be trained on real or simulated data that is collected and can be a multi-modal neural network including a CNN-based architecture such as ResNet, lnceptionV3, CapsuleNet, etc. for analyzing images and a RNN or a TFCNN for analyzing audio signals.

[0053] In an implementation, drowsiness of the vehicle occupant 202 can be detected from data captured from the in-cabin camera. Using the captured data, the first trained neural network, which may be trained by images containing clearly visible faces, detects the vehicle occupant’s inattention and distraction and determines whether the vehicle occupant 202 is drowsy. The first trained neural network in this case may be trained on real or simulated data that is collected and can be a CNN-based architecture such as ResNet, lnceptionV3, CapsuleNet, etc. to analyze images.

[0054] In an implementation, thermal comfort of the vehicle occupant 202 can be estimated from data captured from the in-cabin camera, vehicle data and surrounding environmental data. Using the captured data, the first trained neural network, which may be trained by data from various environmental and vehicle sensors as well as images from the in-cabin camera, analyzes the condition of the vehicle occupant 202 on whether he or she expresses satisfaction with the thermal environment. The first trained neural network then determines a numerical measure of thermal comfort for the vehicle occupant 202. The first trained neural network in this case may be trained on real or simulated data that is collected and can be a multi-modal neural network including a CNN-based architecture such as ResNet, lnceptionV3, CapsuleNet, etc. for analyzing images and machine-learning (ML) techniques such as Random forest, gradient boosting, personalised regression, etc. to analyze numerical data.

[0055] In an implementation, trip comfort of the vehicle occupant 202 can be estimated from data captured from the vehicle and environmental data. Using the captured data, the first trained neural network, which may be trained by data from various environmental and vehicle sensors, evaluates the vehicle occupant’s comfort of the trip with respect to navigation and / or the driver’s driving style. The first trained neural network then determines a numerical measure of trip comfort for the vehicle occupant 202. The first trained neural network in this case may be trained on real or simulated data that is collected and can be a multi-modal neural network including machine-learning techniques such as Random forest, gradient boosting, personalised regression, etc. and a RNN-based architecture such as Long Shortterm Memory (LSTM) network, Gated Recurrent Unit (GRU) etc.

[0056] The first trained neural network can obtain the cumulative score of the vehicle occupant 202 by assigning a weightage score for each of the one or more first level characteristics based on an importance of the first level characteristics to the vehicle occupant 202. This can be accomplished by using a mechanism that assigns different weightages to the first level characteristics according to their importance for the driver and vehicle occupant(s) 202 respectively. For example, the first level characteristics having a high weightage can be assumed to be a predetermined value x and the characteristics having a low weightage can be x-n, where n can be a pre-determined value. A weightage score is then calculated for each first level characteristic and the cumulative score is the sum of the weightage score of each of the first level characteristics.

[0057] In an example embodiment, if the vehicle occupant 202 is the driver, drowsiness detection or distraction detection can have more weightage as compared to body-pose estimation for the driver. A cumulative score is computed and fed to a thresholding block of the first trained neural network. An example of the different weightages of the first level characteristics for a passenger and a driver are shown in Table 1 below. Weightages of First level characteristics Feature Passenger Driver Face recognition High High Emotion recognition High High Age and Gender High Low Pose Estimation High Low Action Recognition High High Mental fatigue Low High Distraction Low High Drowsiness Low High Thermal comfort High High Trip comfort High Low Table 1

[0058] The thresholding block is defined because some of the first level characteristics may not contribute towards calculating the comfort level of the vehicle occupant 202 singularly, and the system 200 therefore does not always have to obtain one or more second level characteristics. For example, if an infant or elderly occupant is sleepy and thermally comfortable, it may already mean that he or she is comfortable. In this way, the system 200 can reduce its computation cost. On the other hand, if the cumulative score calculated from the one or more first level characteristics is above a predetermined threshold value, the relevant first level characteristics are passed to the second neural network module 208 for the next step.

[0059] At step E, when the first trained neural network determines the cumulative score is above the predetermined threshold value, the relevant first level characteristics are sent to the second neural network module 208. The second neural network module 208 is configured to implement a second trained neural network for processing one or more second level characteristics to determine a comfort score of the vehicle occupant. The one or more second level characteristics can be processed by obtaining relevant data from the one or more data capturing modules 204. Alternatively, the one or more second level characteristics can also be processed using the relevant first level characteristics passed from the first neural network module 206. The second trained neural network may be a multi-modal neural network-based occupant comfort modelling system that processes the one or more second level characteristics. The second level characteristics may include at least one of: human interaction, in-cabin comfort, generic emotion and cognitive load.

[0060] Human activity recognition plays a significant role in analysing human-to-human interaction along with individual activities. For example, if the driver is interacting with the infotainment system, he or she could be alerted for being distracted if the interaction with the infotainment system is more than five seconds. In an implementation, human interaction of the vehicle occupant 202 can include individual activities and interactions with other vehicle occupants which can be obtained from the unique identity, emotion recognition, activity recognition and pose estimation that are processed by the first trained neural network. Using the obtained information, the second trained neural network, which may be trained by labelled data with activities and interactions rated from the lowest score of -5 to the highest score of +5, measures the quality of individual activities and interactions. The second trained neural network may subsequently evaluate if the vehicle occupants are interacting within themselves as well as individual activities. The second trained neural network in this case can be trained using machine-learning techniques such as Random forest, gradient boosting, personalised regression along with RNN-based architecture such as LSTM.

[0061] Humans generally express emotions as a response to stimuli. Therefore, evaluating emotions frame-by-frame would not accurately depict the full emotion of a person (or vehicle occupant). On the other hand, by using generic emotion, a person’s emotions can be analysed over a time period and certain impulse reactions can be omitted. In an implementation, generic emotion of the vehicle occupant 202 can be obtained from the unique identity and emotion recognition that were processed by the first trained neural network. Using the obtained information, the second trained neural network, which may be trained by labelled data with emotions, can identify emotional expressions such as anger, disgust, fear, happiness, sadness and surprise of the vehicle occupant 202 and then evaluate the emotion of the vehicle occupant 202 over a period of time. The second trained neural network in this case can be a RNN-based architecture such as LSTM.

[0062] Several complex tasks involve a wide range of cognitive skills, multisensory perception and motor abilities. Cognitive impairment, in its varying degrees, can therefore be an important factor that can affect the ability of the driver, especially older adults, to drive a motor vehicle. In an implementation, cognitive load of the vehicle occupant 202 can be obtained from the unique identity, mental fatigue, distraction detection and drowsiness detection that were processed by the first trained neural network. Using the obtained information, the second trained neural network can evaluate the used amount of working memory resource and determine a measure of cognitive load. The second trained neural network may be trained by labelled data with defined cognitive load based on the inputs from mental fatigue, distraction detection and drowsiness detection from the first trained neural network. The labelled data may have ratings ranging from 0 where there is no cognitive load to +5 where there is maximum cognitive load. The second trained neural network in this case can be a RNN-based architecture such as LSTM and trained with machine learning techniques such as Random forest, gradient boosting and personalised regression.

[0063] Thermal comfort is an important characteristic as the human body is usually relaxed when the environmental temperature is compatible. The human mind is also at relative peace when it knows that the driving is safe and an optimum route has been chosen. In an implementation, the in-cabin comfort of the vehicle occupant 202 can be obtained from the unique identity, thermal comfort and trip comfort that were processed by the first trained neural network. Using the obtained information, the second trained neural network can evaluate the comfort and determine a measure of in-cabin comfort of the vehicle occupant 202. The second trained neural network may be trained by labelled data with defined cognitive load based on the inputs from thermal comfort and trip comfort from the first trained neural network. The labelled data may have ratings ranging from -5 where it is most uncomfortable to +5 where it is the most comfortable. The second trained neural network in this case can be a RNN-based architecture such as LSTM and trained with machine-learning techniques such as Random forest and gradient boosting.

[0064] The second trained neural network may include a probabilistic model and a comfort score predictor that calculates a comfort score for each vehicle occupant 202 based on the processed one or more second level characteristics. At step F, the second neural network module 208 may obtain data from the event log database 214 to calculate the comfort score.

[0065] The event log database 214 can contain history of previous trips which includes information on rest-stop criticality level, distance to rest-stop, location of rest-stop and comfort score. The event log database 214 may also contain current trip information such as the vehicle occupant’s unique identity, age and gender, comfort score, cumulative score and current route information which may be obtained from the event log database 214 by the second neural network module 208. At step G, the second neural network module 208 may also obtain the unique profile of the vehicle occupant 202 from the profile database 212 to calculate the comfort score.

[0066] With every event, the relevant data is fed to the comfort score predictor and using a comfort score predictor algorithm, such as a time-series based neural network, determines a comfort score on a scale of 1 to 5 corresponding to each of the vehicle occupants respectively. Specifically, the second trained neural network may analyse the second level characteristics and historical data of the vehicle occupant to determine the comfort score, where the historical data includes history of previous trips and current trip information of the vehicle occupant 202. At step H in Figure 2, the comfort score is then fed to the recommendation module 210.

[0067] After receiving the comfort score, the recommendation module 210 presents a plurality of rest-stops to the vehicle occupant 202 according to their comfort level, which is based on the comfort score and / or the cumulative score. The recommendation module 210 may include a display monitor to visually present the recommended rest-stops for the vehicle occupant 202 to choose. In an alternative embodiment, the recommendation module 210 may also present the recommended rest-stops verbally to the vehicle occupant 202 and the vehicle occupant 202 can choose the desired rest-stop through verbal communication.

[0068] The recommendation module 210 may present the rest-stops by obtaining the vehicle occupant’s unique identity, age and gender, comfort score, location and duration of previous rest-stops and current route information. Using the obtained data, the recommendation module 210 presents the recommended rest-stops to the vehicle occupant 202 by accessing the rest-stop criticality level (low, medium, high), distance of the vehicle to a rest-stop and the location of a rest-stop. The recommendation module 210 may include a third trained neural network that is trained using the vehicle occupants’ demographic data and a high-definition map. The third trained neural network in this case may be a RNN-based architecture such as LSTM and machine-learning techniques such as Random Forest and gradient boosting.

[0069] The plurality of rest-stops that are presented may be ranked based on a comfort level of the vehicle occupant 202, where the comfort level is based on the criticality level of the rest-stop and distance and location of the rest-stop. The recommended rest-stops may also be ranked based on historical preferences of the vehicle occupant 202 and may also represent the probability of a suitable rest-stop based on the comfort level of the vehicle occupant 202. The recommendation module 210 may also compute a second list of rest-stops to be presented to the vehicle occupant 202.

[0070] In an example embodiment, only the top-ranked rest-stop of the plurality of rest-stops may be presented. The vehicle occupant 202 may not be satisfied with the top-ranked rest stop and the recommendation module 210 presents the second-ranked rest-stop of the plurality of rest-stops. In an alternate embodiment, the entire list of ranked rest-stops may be presented. If the vehicle occupant 202 is not satisfied with any of the presented rest stops, the recommendation module 210 may present the second list of rest-stops. Alternatively, the vehicle occupant 202 can select their preferred rest-stop manually. In such a situation, the recommendation module 210 stores, at step I in Figure 2, the preferred rest stop in the profile database 212 and the event log database 214 for neural network learning and feedback to provide better suggestions in the future.

[0071] The data and feedback collected during the trip can be used for other corrective measures such as vehicle occupant safety features, for example adjust the breaking and steering in rash driving situations, informing healthcare in health-related issues or contacting emergency services in the event of suspicious activities or bad intentions and personalization of HVAC (heating, ventilation and air-conditioning) and HMI (human machine interface) or the like.

[0072] Figure 3 shows a detailed flow chart illustrating an implementation of the method according to the example embodiments. At step 1, audio data, video data, physiological data, vehicle data and route information are captured from an array of sensors, which may include but not limited to in-cabin cameras, GPS sensors, heart-rate sensors, microphones and thermometers. At step 2, the obtained data is sent to the first trained neural network. At step 3, the first trained neural network processes the one or more first level characteristics of the vehicle occupant. The first level characteristics include age, gender, unique identity, emotion recognition, activity recognition, pose estimation, mental fatigue, distraction, drowsiness, thermal comfort and trip comfort. Thermal comfort and trip comfort may be processed by a neural network different from the neural network that processes age, gender, unique identity, emotion recognition, activity recognition, pose estimation, mental fatigue, distraction and drowsiness. At step 4, a unique profile is created for each vehicle occupant and stored into the occupant profile database. At step 5, a cumulative score based on the first level characteristics is determined if it exceeds a predetermined threshold value. If no, the process stops as it means that the vehicle occupant is currently comfortable. If yes, it means that the vehicle occupant is uncomfortable and the cumulative score and first level characteristics are sent to an occupant comfort modelling system (or a second trained neural network) at step 6.

[0073] At step 7, the occupant comfort modelling system processes second level characteristics of human interaction, in-cabin comfort, generic emotion and cognitive load. At step 8, the processed second level characteristics are sent to a probabilistic model. At step 9, the probabilistic model computes the second level characteristics and sends to a comfort score predictor. The probabilistic model may also send the computed second level characteristics to an event log database at step 10. At step 11, the comfort score predictor calculates a comfort score of the vehicle occupant and sends to a recommendation system. The comfort score predictor may also send the calculated comfort score to the event log database at step 12. The recommendation system may obtain event logs from the event log database at step 13. At step 14, the recommendation system recommends a plurality of rest-stops based on the rest-stop criticality level, distance to the rest-stop and location for the rest-stop. The recommended reststops by the recommendation system are stored in the event log database. At step 15, the vehicle occupant determines whether to accept the recommended rest-stops. If they are accepted, the rest-stop preference for the vehicle occupant will be stored in the event log database at step 16. If the vehicle occupant does not accept the recommended rest-stops and requests for another rest-stop suggestion, the vehicle occupant preference is stored in the event log database at step 17. If the vehicle occupant does not accept the recommended rest-stop and manually inputs his or her preferred rest-stop, the preferred rest-stop is stored in the event log database at step 18.

[0074] It will be appreciated by a person skilled in the art that numerous variations 5 and / or modifications may be made to the present invention as shown in the specific embodiments without departing from the spirit or scope of the invention as broadly described. The present embodiments are, therefore, to be considered in all respects to be illustrative and not restrictive. 10

Claims

1. A method (100) for recommending rest-stops to a vehicle occupant (202), comprising:obtaining (102) one or more first level characteristics of the vehicle occupant (202);processing (104), by a first trained neural network, the one or more first level characteristics to obtain a cumulative score of the vehicle occupant (202);characterized in that:obtaining (106) one or more second level characteristics of the vehicle occupant (202);processing (108), by a second trained neural network, the one or more second level characteristics to determine a comfort score of the vehicle occupant (202); andpresenting (110) a plurality of rest-stops to the vehicle occupant (202) based on the cumulative score and the comfort score.

2. The method (100) according to claim 1, wherein processing the one or more first level characteristics comprises:creating a unique profile for the vehicle occupant (202); andassigning a weightage score for each of the one or more first level characteristics based on an importance of the first level characteristics to the vehicle occupant (202);wherein the cumulative score is obtained based on the sum of each of the weightage scores.

3. The method (100) according to claim 1 or 2, wherein the first level characteristics comprises at least one of: age, gender, identity, emotion, activity, pose, mental fatigue, distraction, drowsiness, thermal comfort and trip comfort.

4. The method (100) according to any one of the preceding claims, wherein obtaining the first level characteristics comprises capturing audio, video, physiological data, vehicle data and / or route information.

5. The method (100) according to any one of the preceding claims, wherein obtaining one or more second level characteristics comprises:determining, by the first trained neural network, if the cumulative score is above a predetermined threshold value.

6. The method (100) according to any one of the preceding claims, wherein the second level characteristics comprises at least one of: human interaction, in-cabin comfort, generic emotion and cognitive load.

7. The method (100) according to any one of the preceding claims, wherein processing the one or more second level characteristics to determine a comfort score comprises:analyzing, by the second trained neural network, the second level characteristics and historical data of the vehicle occupant (202).

8. The method (100) according to claim 7, wherein historical data comprises history of previous trips and current trip information of the vehicle occupant (202).

9. The method (100) according to any one of the preceding claims, wherein presenting the plurality of rest stops comprises:ranking the plurality of rest stops based on a comfort level of the vehicle occupant (202), the comfort level comprising criticality level, distance and location of the rest-stop.

10. The method (100) according to any one of the preceding claims, further comprising:storing a preference of the vehicle occupant (202) for neural network learning if the vehicle occupant (202) selects a rest-stop different from the presented plurality of rest-stops; andpresenting a second set of rest-stops to the vehicle occupant (202).

11. A system (200) for recommending rest-stops to a vehicle occupant (202), comprising:one or more data capturing modules (204) configured to obtain one or more first level characteristics and one or more second level characteristics of the vehicle occupant (202);a first neural network module (206) configured to implement a first trained neural network for processing the one or more first level characteristics to obtain a cumulative score of the vehicle occupant (202);characterized in that:a second neural network module (208) configured to implement a second trained neural network for processing the one or more second level characteristics to determine a comfort score of the vehicle occupant (202); anda recommendation module (210) configured to present a plurality of rest-stops to the vehicle occupant (202) based on the cumulative score and the comfort score.

12. The system (200) according to claim 11, wherein the first neural network module (206) is further configured to:create a unique profile for the vehicle occupant (202); andassign a weightage score for each of the one or more first level characteristics based on an importance of the first level characteristics to the vehicle occupant (202); anddetermine if the cumulative score is above a predetermined threshold value;wherein the cumulative score is obtained based on the sum of each of the weightage scores.

13. The system (200) according to any one of claims 11-12, wherein the one or more data capturing modules (204) is further configured to:capture audio, video, physiological data, vehicle data and / or route information.

514. The system (200) according to any one of claims 11-13, wherein the second neural network module (208) is further configured to:analyze the second level characteristics and historical data of the vehicle occupant (202).1015. The system (200) according to any one of claims 11-14, wherein the recommendation module (210) is further configured to:rank the plurality of rest stops based on a comfort level of the vehicle occupant (202), the comfort level comprising criticality level, distance and15 location of the rest-stop;store a preference of the vehicle occupant (202) for neural network learning if the vehicle occupant (202) selects a rest-stop different from the presented plurality of rest-stops; andpresent a second set of rest-stops to the vehicle occupant (202).

Images