Adaptive personal thermal control in vehicles

Abstract

A system in a vehicle includes a personal thermal device. The personal thermal device provides heating or cooling to an individual occupant of the vehicle. The system also includes a controller that implements reinforcement learning to control the personal thermal device. The controller obtains, from one or more sensors, a state indicative of a current condition to obtain a score determined from the state and representing a reward used in the reinforcement learning. The controller provides, using the reinforcement learning, a stochastic policy based on the score, the stochastic policy indicating probabilities of taking particular actions for controlling the personal thermal device, the score serving as feedback for feedback control of the personal thermal device.

Description

Technical Field

[0001] This disclosure relates to adaptive personal thermal control in vehicles. Background Technology

[0002] Vehicles (e.g., automobiles, motorcycles, construction equipment, farm equipment) are increasingly incorporating features for the safety and comfort of occupants (i.e., drivers or passengers). Exemplary safety features include alarms (e.g., lane departure warnings) and semi-autonomous operation (e.g., collision avoidance systems, automatic braking). Exemplary personal heating devices include heated seats. Unlike cabin temperature or other comfort features within a vehicle that serve all occupants, personal heating devices are provided to ensure the comfort of an individual occupant of the vehicle. Therefore, adaptive personal thermal control is desirable in vehicles. Summary of the Invention

[0003] In one exemplary embodiment, a system in a vehicle includes a personal heating device. The personal heating device provides heating or cooling to a single occupant of the vehicle. The system also includes a controller that implements reinforcement learning to control the personal heating device. The controller obtains a state indicating a current condition from one or more sensors. The controller obtains a score determined based on the state and representing a reward used in the reinforcement learning. The controller uses the reinforcement learning to provide a stochastic policy based on the score, the stochastic policy indicating the probability of taking a specific action to control the personal heating device, the score serving as feedback for feedback control of the personal heating device.

[0004] In addition to one or more features described in this article, the state also includes human influence factors (HIF) determined based on the occupant’s manual actions or based on emotion analysis.

[0005] In addition to one or more features described herein, the state determined based on manual action includes the elapsed time or duration since the occupant manually adjusted the personal heating device, or the elapsed time or duration since the occupant entered or started the vehicle.

[0006] In addition to one or more features described herein, the state determined by sentiment analysis includes the influence of the occupant, which is determined by using a camera pointed at the occupant's face or a microphone recording the occupant's speech or the occupant's biometrics.

[0007] In addition to one or more features described herein, the state includes system influence factors (SIFs), which are typically parameters that affect the temperature felt by the occupants.

[0008] In addition to one or more features described herein, the state includes situational influence factors (CIFs) that are not specific to the occupants.

[0009] In addition to one or more features described in this paper, the controller obtains scores by mapping states to scores using a mapping function developed via machine learning.

[0010] In addition to one or more features described in this article, the controller also uses occupant-specific user profiles to obtain scores.

[0011] In addition to one or more of the features described herein, a personal heating device is a heated or ventilated seat for occupants to sit in.

[0012] In addition to one or more features described herein, the occupant is the driver of the vehicle, and the personal heating device is a heated or cooled steering wheel.

[0013] In another exemplary embodiment, a non-transitory computer-readable medium stores instructions that, when processed by one or more processors, implement a method in a vehicle. The method includes obtaining a state indicative of a current condition from one or more sensors, and obtaining a score determined based on the state and representing a reward. The method also includes performing reinforcement learning to control a personal thermal device. The personal thermal device provides heating or cooling to a single occupant of the vehicle, and the reward is used in the reinforcement learning. Based on the score, the reinforcement learning is used to provide a stochastic policy indicating the probability of taking a specific action to control the personal thermal device, the score serving as feedback for feedback control of the personal thermal device.

[0014] In addition to one or more features described herein, obtaining state includes obtaining human influence factors (HIF) determined based on occupant manual actions or emotion analysis.

[0015] In addition to one or more features described herein, obtaining the state as HIF determined based on manual action includes obtaining the elapsed time or duration since the occupant manually adjusted the personal heating device or the elapsed time or duration since the occupant entered or started the vehicle.

[0016] In addition to one or more features described herein, obtaining a state as HIF determined based on sentiment analysis includes obtaining the occupant's influence, which is determined by using a camera pointed at the occupant's face or a microphone recording the occupant's speech or the occupant's biometrics.

[0017] In addition to one or more features described in this article, obtaining the state includes obtaining system influence factors (SIFs), which are typically parameters that affect the temperature felt by the occupants.

[0018] In addition to one or more features described herein, obtaining state includes obtaining contextual influencing factors (CIFs) that are not specific to the occupant.

[0019] In addition to one or more features described in this paper, obtaining scores involves mapping states to scores using a mapping function developed via machine learning.

[0020] In addition to one or more features described in this article, obtaining a score also includes using occupant-specific user profiles.

[0021] In addition to one or more of the features described herein, a personal heating device is a heated or ventilated seat for occupants to sit in.

[0022] In addition to one or more features described herein, the occupant is the driver of the vehicle, and the personal heating device is a heated or cooled steering wheel.

[0023] The above-described features and advantages, as well as other features and advantages of this disclosure, will become apparent when taken in conjunction with the accompanying drawings and the following detailed description. Attached Figure Description

[0024] Its features, advantages, and details are presented by way of example only in the following detailed specification, which refers to the accompanying drawings, wherein:

[0025] Figure 1 A vehicle including adaptive personal thermal control according to one or more embodiments is shown;

[0026] Figure 2 This is a process flow of a method for performing adaptive personal thermal control in a vehicle according to one or more embodiments; and

[0027] Figure 3 It is a process flow of a method for determining an experience quality score as part of a process of performing adaptive personal thermal control in a vehicle, according to one or more embodiments. Detailed Implementation

[0028] The following description is exemplary in nature only and is not intended to limit this disclosure, its application, or use. It should be understood that throughout the drawings, corresponding reference numerals denote the same or corresponding parts and features.

[0029] Embodiments of the systems and methods detailed herein relate to adaptive personal thermal control in vehicles. As previously mentioned, a personal thermal system refers to a system designed for the comfort of an individual occupant. Exemplary personal thermal systems include heated seats, ventilated seats, heated steering wheels, and cooled steering wheels. Existing methods for controlling personal thermal systems involve occupants making manual selections or automatic controls based on predefined lookup tables. However, the initial selection may no longer be comfortable, or the occupant may forget to activate the personal thermal system. According to one or more embodiments detailed herein, control of one or more personal thermal systems is automatically performed based on determining the corresponding occupant's Quality of Experience (QoX) score.

[0030] According to an exemplary embodiment, Figure 1 A vehicle 100 including adaptive personal thermal control is shown. Figure 1 The exemplary vehicle 100 shown is an automobile 101. An exemplary adaptive personal thermal system 110 of vehicle 100 includes one or more seats 120, a steering wheel 130, and a controller 140. That is, one or more seats 120 and steering wheel 130 may include heating elements (not shown) based on radiant heat from an electric current. One or more seats 120 may additionally or alternatively include a fan or blower (not shown) that provides cooling to the surfaces of the seat 120. Steering wheel 130 may include a blower or thermoelectric reversible heat pump (not shown) to heat or cool fluid circulating therethrough. While seats 120 and steering wheel 130 for the exemplary adaptive personal thermal system 110 are shown, alternative or additional adaptive personal thermal devices 125 corresponding to occupants and capable of controllable heating or cooling may also be adaptively controlled.

[0031] According to one or more embodiments, controller 140 can perform adaptive individual thermal control. That is, controller 140 can control the current through the heating element based on the determination of the QoX score of the affected occupant, or it can control a fan, blower, or heat pump, as referenced. Figure 2 As discussed. For example, if the occupant is the driver of vehicle 100, controller 140 can control the heating elements or fan in driver's seat 120 and steering wheel 130 based on the determination of the driver's QoX score. The determination of the QoX score may involve one or more sensors 150 (e.g., temperature sensor, humidity sensor, window position sensor, sunroof position sensor, vehicle speed sensor).

[0032] Controller 140 may include processing circuitry, which may include application-specific integrated circuits (ASICs), electronic circuitry, a processor (shared, dedicated, or grouped) and memory executing one or more software or firmware programs, combinational logic circuitry, and / or other suitable components providing the described functionality. Controller 140 may include a non-transitory computer-readable medium storing instructions that, when processed by one or more processors of controller 140, implement a method for performing adaptive personal thermal control in vehicle 100 according to one or more embodiments detailed herein.

[0033] Figure 2 This is a process flow of a method 200 for performing adaptive personal thermal control in a vehicle 100 according to one or more embodiments. Figure 2The processing flow shown can be repeated (i.e., performed individually) for each adaptive personal heating device 125 (e.g., seat 120, steering wheel 130) for each occupant. An exemplary adaptive personal heating system 110 is shown including a heated seat 120 and a heated steering wheel 130. As previously mentioned, alternative or additional features (e.g., ventilated seat 120) and adaptive personal heating devices 125 may be part of the adaptive personal heating system 110. At box 210, a QoX score is determined. Reference Figure 3 The process involved in this determination is described in detail. As indicated, the QoX score can be provided as a reward r corresponding to an iteration time step t from the set of possible rewards R. t An exemplary set R of rewards and corresponding scores could be: {Comfortable (+1), Too Hot (-1), Hot (-0.5), Cold (-0.5), Too Cold (-1)}.

[0034] For reference Figure 3 As shown and discussed, the determination of the QoX score at box 210 can use information about the current adaptive personal thermal system 110 obtained from sensor 150. As indicated, this information can be the current state s corresponding to an iterative time step t from the set of possible states S. t The form of the state set S. An exemplary state set S may include {seat thermocouple readings, interior air temperature, ambient temperature, air cooling effect, interior humidity, ambient humidity, position of each window, sunroof position, vehicle speed, and solar angle}. The state set S may also include states based on occupant impact (such as {negative, neutral, positive}) or contextual information, such as references. Figure 3 Further discussion is needed. (See reference) Figure 3 The camera 305 discussed can be used to obtain some of these states, such as the impact of the occupants.

[0035] An occupant profile 220, developed and stored for a specific occupant, can be used to determine a QoX score. This occupant profile 220 can be updated after each execution of method 200 for the occupant. Occupants can be identified in various ways. For example, if the occupant is the driver, a key card carried by the occupant can be used to identify the driver. As another example, an occupant can be identified based on their cellular device or other personal wireless device (e.g., a wearable device) connected to the infotainment system of vehicle 100. The determination of the QoX score at box 210 provides a quantitative value for implementing a machine learning technique called reinforcement learning (RL) at box 230.

[0036] The RL at box 230 can be model-free, where learning is directly based on experience and iteration. An exemplary implementation of the RL at box 230 could involve time-difference algorithms, such as the Asynchronous Advantage Actor-Commentator (A3C). The RL at box 230 provides a stochastic policy π. θ (a|s) represents the probability of taking a specific action given a state s and parameters θ. In other words, for each iteration of method 200, controller 140 performs feedback control on the given adaptive personal thermal device 125. Specifically, based on the state s at the current time step t, using the QoX score (i.e., reward) as feedback. t The action a is mapped to be implemented by controller 140. t The parameter θ is a known parameter of RL, such as the activation function and weights.

[0037] At box 240, based on the random policy provided by RL at box 230, one or more actions from the set A of possible actions are selected. These actions a t Implemented by controller 140. A set of exemplary actions includes {off, heat level 1, heat level 2, heat level 3, ventilation level 1, ventilation level 2, ventilation level 3}. The set A of possible actions can vary based on the adaptive personal heating device 125 and its capabilities. For example, if the steering wheel 130 can be heated but not cooled, possible action a... t The set A can include {off, heat level 1, heat level 2, heat level 3} but without cooling action. Figure 2 The process shown can be performed iteratively (e.g., periodically) for each adaptive personal thermal device 125 in use.

[0038] Figure 3 This is a process flow of a method 300 for determining a QoX score as part of a process for performing adaptive individual thermal control in a vehicle 100, according to one or more embodiments. Figure 3 The various factors considered in generating the QoX score are shown. These factors include the Systemic Influence Factor (SIF) identified in box 310, the Human Influence Factor (HIF) identified in box 330, and the Contextual Influence Factor (CIF) identified in box 340. Each factor includes a parameter that can be represented as a numerical (e.g., 0, 1) or categorical (e.g., good, bad, okay) value. The figure shows the different states s at the current iteration timestamp. t Different processes are used for method 300. At box 210, parameters that are part of SIF, HIF and CIF are mapped to QoX scores according to a mapping function developed via machine learning (i.e., by training a machine learning algorithm using data).

[0039] The System Influence Factor (SIF) determined at box 310 is the state s obtained from sensor 150 of vehicle 100 as the current iteration timestamp. t Some parameters. Typically, states s classified as SIF... t This can be considered as a state affecting the temperature experienced by the occupants. Sensor 150 is known and typically existing, so it does not need to be added for adaptive individual thermal control. As previously mentioned, exemplary parameters include {seat thermocouple readings, interior air temperature, ambient temperature, interior humidity, ambient humidity, position of each window, sunroof position, vehicle speed, and solar angle}.

[0040] The Human Influencing Factors (HIF) identified in box 330 are based on occupant manual actions (MA) identified in box 320 and sentiment analysis (SA) identified in box 325. MA and SA utilize some states from the current iteration timestamp. t As indicated. The manual action (MA) observed at box 320 refers to the adjustment performed by the occupant of the controlled adaptive personal heating device 125. As previously stated, according to Figure 2 and Figure 3 The processing flow is implemented for a given adaptive personal thermal device 125 for a given occupant. Therefore, at block 320, obtaining the parameters corresponding to the manual action refers to those parameters executed by the given occupant for the given adaptive personal thermal device 125. Exemplary observation parameters associated with the manual action (i.e., states s) t This includes the elapsed time or duration of the occupant's last setting adjustment or setting override (e.g., time since the given occupant added the thermal setting on seat 120), the elapsed time or duration of the last key cycle (e.g., time since the given occupant last started vehicle 100), and the elapsed time or duration of the time since occupancy (e.g., time since the given occupant entered vehicle 100).

[0041] At box 325, sentiment analysis refers to determining verbal and nonverbal feedback provided by occupants. As previously mentioned, the state set S can include states s based on occupant influence. t Such as {negative, neutral, positive} or contextual information. For example, these states can be determined using a sensor 150 such as a camera 305 and a microphone 307. t A camera 305, shown pointing at the face of an occupant (e.g., the driver), can perform temperature screening (e.g., obtaining the temperature of the occupant's face) and image processing (e.g., analyzing the occupant's facial expressions). An additional sensor 150 can acquire biometrics (e.g., body temperature, heart rate).

[0042] As another part of the sentiment analysis at box 325, microphone 307 (which may be part of an infotainment system for Bluetooth connectivity operation of cellular devices) may be used, for example, to obtain verbal input from the occupant. Known sentiment analysis algorithms can be used to map phrases (e.g., “I’m freezing,” “It’s too hot”) to defined sentiment states. t For example, {negative, neutral, positive}. At box 330, the Human Influence Factor (HIF) is compiled using parameters obtained from manual actions at box 320 and sentiment analysis performed at box 325. At box 330, the HIF may additionally include parameters from the occupant profile 220 for a given occupant. These parameters may include physiological parameters, habits, or updated preferences, which are obtained for a given adaptive personal heating device 125 and a given occupant during each use of the adaptive personal heating system 110.

[0043] In addition to the parameters from Box 310 as part of the Systemic Influence Factor (SIF) and the parameters from Box 330 as part of the Human Influence Factor (HIF), the parameters from Box 340 as part of the Contextual Influence Factor (CIF) are also obtained as state s. t In order to determine the QoX score at box 210. CIF can generally be considered as a state not specific to a given crew member. t Exemplary parameters may relate to the physical context of driving (e.g., weather and traffic conditions), time context (e.g., day, month, time), economic context (e.g., trip cost from an energy perspective), social context (e.g., number of passengers), or task context (e.g., manual control, which may distract the driver and affect perceived quality).

[0044] While the foregoing disclosure has been described with reference to exemplary embodiments, those skilled in the art will understand that various changes can be made and elements can be substituted with equivalents without departing from its scope. Furthermore, many modifications can be made to adapt particular situations or materials to the teachings of this disclosure without departing from the basic scope of this disclosure. Therefore, this disclosure is not intended to be limited to the specific embodiments disclosed, but will include all embodiments falling within its scope.

Claims

1. A system in a vehicle, the system comprising: A personal heating device that provides heating or cooling to a single occupant of the vehicle; as well as A controller that implements reinforcement learning to control the personal thermal device is configured to obtain a state indicating a current condition from one or more sensors, obtain a score determined based on the state and representing a reward used in the reinforcement learning, and use the reinforcement learning to provide a stochastic policy based on the score, the stochastic policy indicating the probability of taking an action to control the personal thermal device, the score serving as feedback for feedback control of the personal thermal device; The states mentioned include human influence factors (HIF) determined based on the occupant's manual adjustment of personal heating devices and emotion analysis based on verbal and nonverbal feedback provided by the occupant, systemic influence factors (SIF) as parameters affecting the temperature felt by the passenger, and situational influence factors (CIF) not specific to the occupant, wherein the emotion analysis includes mapping verbal expressions to defined influence states, and wherein the defined influence states include positive, neutral, and negative.

2. The system according to claim 1, wherein, The state includes human influence factors (HIF) determined based on the occupant's manual actions or based on emotion analysis. The state determined based on manual actions includes the passage of time or duration since the occupant manually adjusted their personal heating device, or the passage of time or duration since the occupant entered or started the vehicle. The state determined based on emotion analysis includes the occupant's influence, which is determined by using a camera pointed at the occupant's face or a microphone recording the occupant's speech or the occupant's biometrics.

3. The system according to claim 1, wherein, The controller is configured to obtain the score by mapping the state to the score using a mapping function developed via machine learning, and also by using a user profile specific to the occupant.

4. The system according to claim 1, wherein, The personal heating device is a heated or ventilated seat where the occupant sits, or a heated or cooled steering wheel.

5. A non-transitory computer-readable medium storing instructions that, when processed by one or more processors, implement a method in a vehicle, the method comprising: The state is obtained from one or more sensors, indicating the current situation, wherein obtaining the state includes obtaining the state of human influence factors (HIF) determined based on the occupant’s manual adjustment of personal heating devices and emotional analysis based on verbal and nonverbal feedback provided by the occupant, system influence factors (SIF) as parameters affecting the temperature felt by the passenger, and situational influence factors that are not specific to the occupant. Obtain a score that is determined based on the stated state and represents a reward; Reinforcement learning is performed to control a personal heating device, wherein the personal heating device provides heating or cooling to a single occupant of the vehicle, and the reward is used in the reinforcement learning; and The reinforcement learning is used to provide a stochastic policy based on the score, the stochastic policy indicating the probability of taking an action to control the personal thermal device, the score serving as feedback for feedback control of the personal thermal device, wherein the sentiment analysis includes mapping verbal expressions to defined influence states, and wherein the defined influence states include positive, neutral, and negative.

6. The non-transitory computer-readable medium according to claim 5, wherein, Obtaining the state includes obtaining human influence factors (HIF) determined based on the occupant's manual actions or based on emotion analysis. Obtaining a state as an HIF determined based on manual actions includes obtaining the elapsed time or duration since the occupant manually adjusted the personal heating device or since the occupant entered or started the vehicle. Obtaining a state as an HIF determined based on emotion analysis includes obtaining the occupant's influence, which is determined by using a camera pointed at the occupant's face or a microphone recording the occupant's speech or the occupant's biometrics.

7. The non-transitory computer-readable medium according to claim 5, wherein, Obtaining the score includes mapping the state to the score using a mapping function developed via machine learning, and obtaining the score also includes using a user profile specific to the occupant.

8. The non-transitory computer-readable medium according to claim 5, wherein, The personal heating device is a heated or ventilated seat where the occupant sits, or a heated or cooled steering wheel.

Images