In-vehicle systems

Abstract

To provide an on-vehicle system that is able to improve the accuracy of the prediction of a future emotion of an occupant.SOLUTION: An on-vehicle system according to the present invention incudes: a control device; an in-vehicle sensor that detects a seating position of an occupant present in a vehicle; a spatial information acquisition device that acquires spatial information in the vehicle; a measurement data acquisition device that acquires measurement data of the occupant; and a storage device that stores a learned model for predicting a future emotion of the occupant; wherein the control device is configured such that, using the learned model, the future feeling of the occupant is predicted from the seating position detected by the in-vehicle sensor, the spatial information acquired by the spatial information acquisition device, and the measurement data acquired by the measurement data acquisition device, and vehicle control is executed in accordance with the predicted future feeling of the occupant.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an in-vehicle system. [Background technology]

[0002] Patent Document 1 proposes a technology for estimating the emotions of a passenger based on biological information obtained from a wearable device. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2018-083583 Summary of the Invention [Problem to be solved by the invention]

[0004] Image data and biometric information directly reflect the emotions of an occupant. Image data and biometric information are effective for estimating the emotions of an occupant at a given time. However, the characteristics expressed in the image data and biometric information may have little or no correlation with future emotions. Therefore, it is difficult to accurately predict the future emotions of an occupant from the image data and biometric information.

[0005] The present invention has been made in view of the above-mentioned problems, and an object of the present invention is to provide an in-vehicle system that can improve the accuracy of predicting the future emotions of an occupant. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems and achieve the object, an in-vehicle system according to the present invention is an in-vehicle system including a control device, an in-vehicle sensor that detects the seating positions of occupants present in the vehicle, a spatial information acquisition device that acquires spatial information about the vehicle interior, a measurement data acquisition device that acquires measurement data of the occupant, and a storage device that stores a trained model for predicting future emotions of the occupant, the measurement data is facial expression of the occupant or biological information of the occupant,The control device is configured to use the learned model to predict the future emotions of the occupant from the seating position detected by the in-vehicle sensor, the spatial information acquired by the spatial information acquisition device, and the measurement data acquired by the measurement data acquisition device, and to execute vehicle control in accordance with the predicted future emotions of the occupant.

[0007] The in-vehicle system according to the present invention can improve the accuracy of predicting the future emotions of the occupant.

[0008] Further, in the above, the spatial information is CO2 concentration information, the vehicle control is air conditioning control that circulates air inside the vehicle or ventilates the vehicle, the trained model is generated by machine learning to derive a predicted result of the occupant's future emotions from the seating position, the CO2 concentration information, and the measurement data, and predicting the occupant's future emotions using the trained model may be configured by providing the seating position detected by the in-vehicle sensor, the CO2 concentration information acquired by the spatial information acquisition device, and the measurement data acquired by the measurement data acquisition device to the trained model, and performing calculation processing of the trained model to obtain a predicted result of the occupant's future emotions from the trained model.

[0009] This makes it possible to predict the future emotions of occupants based on CO2 concentration information inside the vehicle.

[0010] Further, in the above, the spatial information is odor information, the vehicle control is air conditioning control that circulates air inside the vehicle or ventilates the vehicle, and the trained model is generated by machine learning to derive a predicted result of the occupant's future emotions from the seating position, the odor information, and the measurement data, and predicting the occupant's future emotions using the trained model may be configured to provide the seating position detected by the in-vehicle sensor, the odor information acquired by the spatial information acquisition device, and the measurement data acquired by the measurement data acquisition device to the trained model, and to perform calculation processing of the trained model to obtain a predicted result of the occupant's future emotions from the trained model.

[0011] This makes it possible to predict the future emotions of the occupants based on the scent information inside the vehicle.

[0012] Further, in the above, the spatial information is temperature information, the vehicle control is air conditioning control that adjusts the temperature inside the vehicle, and the trained model is generated by machine learning to derive a result of predicting the occupant's future emotions from the seating position, the temperature information, and the measurement data, and predicting the occupant's future emotions using the trained model may be configured to provide the seating position detected by the in-vehicle sensor, the temperature information acquired by the spatial information acquisition device, and the measurement data acquired by the measurement data acquisition device to the trained model, and to perform calculation processing of the trained model to obtain a result of predicting the occupant's future emotions from the trained model.

[0013] This makes it possible to predict the future emotions of the occupants based on the temperature information inside the vehicle.

[0014] Further, in the above, the spatial information is solar radiation information, the vehicle control is solar radiation control that adjusts the solar radiation inside the vehicle, the trained model is generated by machine learning to derive a result of predicting the occupant's future emotions from the seating position, the solar radiation information, and the measurement data, and predicting the occupant's future emotions using the trained model may be configured to provide the seating position detected by the in-vehicle sensor, the solar radiation information acquired by the spatial information acquisition device, and the measurement data acquired by the measurement data acquisition device to the trained model, and to perform calculation processing of the trained model to obtain a result of predicting the occupant's future emotions from the trained model.

[0015] This makes it possible to predict the future emotions of the occupants based on the solar radiation information inside the vehicle. [Effects of the Invention]

[0016] The in-vehicle system according to the present invention has an effect of improving the accuracy of predicting the future emotions of the occupant. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a diagram showing an outline of a vehicle equipped with an in-vehicle system according to a first embodiment. [Figure 2] 1 is a block diagram showing a schematic configuration of an in-vehicle system 2 according to a first embodiment. [Figure 3] 4 is a flowchart showing an example of control performed by a control device in the in-vehicle system according to the first embodiment. [Figure 4] FIG. 10 is a block diagram showing a schematic configuration of an in-vehicle system according to a second embodiment. [Figure 5] 10 is a flowchart showing an example of control performed by a control device in an in-vehicle system according to a second embodiment. [Figure 6] FIG. 10 is a block diagram showing a schematic configuration of an in-vehicle system according to a third embodiment. [Figure 7]10 is a flowchart showing an example of control performed by a control device in an in-vehicle system according to a third embodiment. [Figure 8] FIG. 10 is a block diagram showing a schematic configuration of an in-vehicle system according to a fourth embodiment. [Figure 9] 10 is a flowchart showing an example of control performed by a control device in an in-vehicle system according to a fourth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0018] (Embodiment 1) A first embodiment of an in-vehicle system according to the present invention will be described below, although the present invention is not limited to this embodiment.

[0019] Fig. 1 is a diagram showing an outline of a vehicle 1 equipped with an in-vehicle system 2 according to embodiment 1. Fig. 2 is a block diagram showing a schematic configuration of the in-vehicle system 2 according to embodiment 1.

[0020] 1, a vehicle 1 according to the embodiment includes an in-vehicle system 2, a steering wheel 4, front seats 31 and 32, and a rear seat 33. An arrow A in FIG. 1 indicates the traveling direction of the vehicle 1.

[0021] Occupants 10A, 10B, and 10C are seated in the front seats 31, 32 and the rear seat 33, respectively. Occupant 10A, seated in the front seat 31 opposite the steering wheel 4, is the driver of the vehicle 1. In the following description, when there is no need to distinguish between occupants 10A, 10B, and 10C, they will simply be referred to as occupant 10.

[0022] The in-vehicle system 2 is configured with a control device 21, a storage device 22, an in-vehicle camera 23, an operation panel 24, and the like.

[0023] The control device 21 is configured, for example, by an integrated circuit including a CPU (Central Processing Unit). The control device 21 is communicably connected to the storage device 22, the in-vehicle camera 23, and the operation panel 24. The control device 21 executes programs stored in the storage device 22. The control device 21 also acquires image data from the in-vehicle camera 23, for example.

[0024] The storage device 22 includes, for example, at least one of a ROM (Read Only Memory), a RAM (Random Access Memory), an SSD (Solid State Drive), and an HDD (Hard Disk Drive). The storage device 30 does not need to be a single physical element, but may include multiple elements that are physically separated from one another. The storage device 22 stores programs executed by the control device 21, etc. The storage device 22 also stores various data used when executing programs such as a trained model for predicting the future emotions of the occupant 10 and a trained model for vehicle control, which will be described later.

[0025] 1, the in-vehicle camera 23 is an imaging device arranged in a position inside the vehicle where it can capture images of multiple occupants 10A, 10B, and 10C. The in-vehicle camera 23 functions as an in-vehicle sensor for detecting, for example, a target occupant 10 whose emotion is to be predicted from the multiple occupants 10A, 10B, and 10C inside the vehicle, and outputs image data as sensor data. The image data captured by the in-vehicle camera 23 is transmitted to the control device 21 and temporarily stored in the storage device 22.

[0026] The operation panel 24 is an input / output device such as a touch panel display provided near the driver's seat, and receives operation instructions from the occupant 10 such as the driver, and provides information to the occupant 10.

[0027] The CO2 concentration information acquisition device 25 includes a CO2 concentration sensor, an infrared sensor, etc., and acquires information about the CO2 concentration inside the vehicle. Furthermore, the CO2 concentration information acquisition device 25 can acquire information about the CO2 concentration near the seating position of the occupant 10 in the front seats 31, 32 and the rear seat 33 using vehicle structure information that is information specific to the vehicle 1.

[0028] The air conditioner 5 includes an air compressor and a heater, and adjusts the temperature inside the vehicle by operating the air compressor or heater in accordance with a control signal from the control device 21. For example, when the air conditioner 5 is activated, the control device 21 controls the air conditioner 5 so that the temperature inside the vehicle reaches a target temperature. The target temperature may be a preset temperature or a temperature set by the most recent user. The air conditioner 5 is also configured to be able to introduce outside air into the vehicle interior, thereby enabling ventilation inside the vehicle. The air conditioner 5 is also able to perform air conditioning control, such as blowing air to prioritize and optimize the CO2 concentration at specific locations inside the vehicle, using vehicle structure information, which is information unique to the vehicle 1.

[0029] The control device 21 can detect, for example, the attributes and facial expressions of the occupant 10, the seating position (riding position) of the occupant 10, and the like, based on image data captured by the in-vehicle camera 23. That is, the control device 21 can make a determination using AI (Artificial Intelligence) with a trained model that has been machine-learned to learn the attributes, facial expressions, and seating position (riding position) of the occupant 10 based on the image data. The control device 21 can also make a determination using AI with a trained model that has been machine-learned to learn the attributes, facial expressions, and seating position (riding position) of the occupant 10, based on the seating position of the occupant 10, CO2 concentration information, and measurement data of the occupant 10. The measurement data of the occupant 10 includes, for example, the facial expressions of the occupant 10 captured by the in-vehicle camera 23 and biometric information such as heart rate and blood pressure detected by a wearable device worn by the occupant 10. The control device 21 can also make a determination using AI with a trained model that has been machine-learned to learn the details of vehicle control based on the predicted future emotions of the occupant 10.

[0030] The trained model for predicting future emotions is trained, for example, by supervised learning according to a neural network model, to output emotion estimation results from input data. The trained model for predicting future emotions is generated by repeatedly executing a learning process using a training dataset, which is a combination of input and result data. The training dataset in the trained model for predicting future emotions includes multiple pieces of training data in which input data, such as the seating position of the occupant 10, CO2 concentration information, and measurement data of the occupant 10, are labeled with an output prediction of the occupant's future emotions. The labeling of the input data with a prediction of the future emotions is performed, for example, by a person skilled in the art. In this way, the trained model for predicting future emotions trained using the training dataset receives input data and outputs a prediction of the occupant's future emotions by executing arithmetic processing of the trained model.

[0031] The trained model for vehicle control is trained, for example, by supervised learning according to a neural network model, to output vehicle control results from input data. The training dataset in the trained model for vehicle control includes a plurality of training data in which output vehicle control contents are labeled for input data such as the seating position of the occupant 10, CO2 concentration information, and a prediction of the occupant 10's future emotions, which are given as input. The labeling of the vehicle control contents for the input data is performed, for example, by a person skilled in the art. In this way, the trained model for vehicle control trained using the training dataset outputs vehicle control contents when it receives input data. For example, the vehicle control contents may include air conditioning control, such as air circulation or ventilation, when it is predicted that the occupant 10 will become uncomfortable in the future based on CO2 concentration information.

[0032] Note that the control device 21 is not limited to determining the content of vehicle control using a trained model for vehicle control. For example, the control device 21 may determine the content of vehicle control from CO2 concentration information and a prediction of the future emotion of the occupant 10 based on a rule that associates the CO2 concentration information inside the vehicle, the prediction of the future emotion of the occupant 10, and the content of vehicle control.

[0033] The control device 21 identifies the seating position (riding position) of the occupant whose emotion is to be predicted based on the detection results of in-vehicle sensors such as the in-vehicle camera 23 and seating sensors. That is, the control device 21 detects the seating position of the target occupant by seating sensors provided in the front seats 31, 32 and the rear seat 33, respectively, or by image processing of image data captured by the in-vehicle camera 3. In addition, in the in-vehicle system 2 according to the first embodiment, the seat positions in the vehicle may be displayed on the display of the operation panel 24, and the seat position where the target occupant is sitting may be specified by the occupant 10, such as the driver.

[0034] In the in-vehicle system 2 according to the first embodiment, when it is predicted that the target occupant will become uncomfortable in the future based on the CO2 concentration information inside the vehicle, the control device 21 performs air conditioning control such as air circulation and ventilation using the air conditioner 5. Note that, as the vehicle control, the control device 21 may perform control to open an openable window provided in the vehicle 1 to ventilate the interior of the vehicle, instead of the air conditioning control by the air conditioner 5.

[0035] FIG. 3 is a flowchart showing an example of control performed by the control device 21 in the in-vehicle system 2 according to the first embodiment.

[0036] First, in step S1, the control device 21 identifies the seating position of the target occupant based on the detection results of the in-vehicle sensors. Next, in step S2, the control device 21 acquires CO2 concentration information, which is spatial information about the interior of the vehicle. Next, in step S3, the control device 21 acquires measurement data of the target occupant, such as facial expression, from image data of the in-vehicle camera 23. Next, in step S4, the control device 21 determines whether or not the target occupant's emotional deterioration is predicted based on the target occupant's seating position, the CO2 concentration information in the vehicle, and the measurement data of the target occupant. If the control device 21 determines that the target occupant's emotional deterioration is not predicted, the control device 21 determines No in step S4 and terminates the series of controls without changing the in-vehicle environment. If the control device 21 determines that the target occupant's emotional deterioration is predicted, the control device 21 determines Yes in step S4 and proceeds to step S5. In step S5, the control device 21 uses a trained model for predicting future emotions to predict the future emotions of the target occupant based on the target occupant's seating position, CO2 concentration information inside the vehicle, and measurement data of the target occupant. Next, in step S6, the control device 21 uses a trained model for vehicle control to determine the content of air conditioning control, such as air circulation and ventilation inside the vehicle, based on the target occupant's seating position, CO2 concentration information inside the vehicle, and the predicted future emotions of the target occupant. Next, in step S7, the control device 21 executes air conditioning control based on the determined content of air conditioning control.

[0037] As a result, the in-vehicle system 2 according to the first embodiment can prevent the target occupant from feeling uncomfortable in the future due to a deterioration in the in-vehicle environment, such as an increase in the CO2 concentration in the vicinity of the seating position of the target occupant.

[0038] (Embodiment 2) Hereinafter, a second embodiment of the in-vehicle system according to the present invention will be described, with the explanation of the contents common to the first embodiment being omitted as appropriate.

[0039] FIG. 4 is a block diagram showing a schematic configuration of an in-vehicle system 2 according to the second embodiment.

[0040] In the in-vehicle system 2 according to the second embodiment, the spatial information about the interior of the vehicle is odor information about the interior of the vehicle, and the system is provided with an odor information acquisition device 26 for acquiring the odor information about the interior of the vehicle. The odor information acquisition device 26 includes, for example, an odor sensor and acquires odor information about the interior of the vehicle. The odor information includes, for example, odor information about cosmetics and odor information about food. Furthermore, the odor information acquisition device 26 can acquire odor information about the vicinity of the seating position of the occupant 10 in the front seats 31, 32 and the rear seat 33 using vehicle structure information that is information unique to the vehicle 1.

[0041] The control device 21 predicts the future emotions of the occupant 10 using a trained model for predicting future emotions based on the seating position of the occupant 10, odor information inside the vehicle, and measurement data of the occupant 10. If it is predicted that the occupant 10 will become uncomfortable in the future, the control device 21 performs air conditioning control, such as air circulation and ventilation inside the vehicle, as vehicle control using the trained model for vehicle control.

[0042] The training data set in the trained model for predicting future emotions includes multiple training data labeled with a prediction of the future emotions of the occupant 10 as output for input data such as the seating position of the occupant 10, odor information, and measurement data of the occupant 10.

[0043] The training data set in the trained model for vehicle control includes multiple pieces of training data in which the vehicle control content to be output is labeled for input data such as the seating position of the occupant 10, odor information inside the vehicle, and predicted future emotions of the occupant 10. The vehicle control content may, for example, be air conditioning control such as air circulation and ventilation when it is predicted that the occupant 10 will become uncomfortable in the future based on odor information inside the vehicle.

[0044] In the in-vehicle system 2 according to the second embodiment, when it is predicted that the target occupant will become uncomfortable in the future based on odor information about the interior of the vehicle, the control device 21 performs air conditioning control such as air circulation and ventilation using the air conditioner 5. The air conditioner 5 can perform air conditioning control such as blowing air to prioritize and optimize odors at specific locations within the vehicle using vehicle structure information that is information unique to the vehicle 1. Furthermore, the control device 21 may control odors within the vehicle by controlling the opening and closing of openable windows provided in the vehicle 1, instead of the air conditioning control by the air conditioner 5.

[0045] FIG. 5 is a flowchart showing an example of control performed by the control device 21 in the in-vehicle system 2 according to the second embodiment.

[0046] First, in step S11, the control device 21 identifies the seating position of the target occupant, who is the occupant 10 for whom future emotions are predicted, based on the detection results of the in-vehicle sensors. Next, in step S12, the control device 21 acquires odor information, which is spatial information about the interior of the vehicle. Next, in step S13, the control device 21 acquires measurement data of the target occupant, such as facial expressions, from image data of the in-vehicle camera 23. Next, in step S14, the control device 21 determines whether or not a deterioration in the target occupant's emotions is predicted based on the seating position of the target occupant, the odor information about the interior of the vehicle, and the measurement data of the target occupant. If the control device 21 determines that a deterioration in the target occupant's emotions is not predicted, the control device 21 determines No in step S14 and terminates the series of controls without changing the in-vehicle environment. On the other hand, if the control device 21 determines that a deterioration in the target occupant's emotions is predicted, the control device 21 determines Yes in step S14 and proceeds to step S15. In step S15, the control device 21 uses a trained model for predicting future emotions to predict the future emotions of the target occupant based on the target occupant's seating position, odor information inside the vehicle, and measurement data of the target occupant. Next, in step S16, the control device 21 uses a trained model for vehicle control to determine the content of air conditioning control, such as air circulation and ventilation inside the vehicle, based on the target occupant's seating position, odor information inside the vehicle, and the predicted future emotions of the target occupant. Next, in step S17, the control device 21 executes air conditioning control based on the determined content of air conditioning control.

[0047] As a result, the in-vehicle system 2 according to the second embodiment can prevent the target occupant from feeling uncomfortable in the future due to a deterioration in the in-vehicle environment, such as an increase in the intensity of odors around the seating position of the target occupant.

[0048] (Embodiment 3) Hereinafter, an in-vehicle system according to a third embodiment of the present invention will be described. Note that the description of the content common to the first embodiment of the third embodiment will be omitted as appropriate.

[0049] FIG. 6 is a block diagram showing a schematic configuration of an in-vehicle system 2 according to the third embodiment.

[0050] In the in-vehicle system 2 according to the third embodiment, the spatial information about the interior of the vehicle is temperature information about the interior of the vehicle, and the in-vehicle system 2 is provided with a temperature information acquisition device 27 for acquiring the temperature information about the interior of the vehicle. The temperature information acquisition device 27 includes, for example, a temperature sensor, and acquires temperature information about the interior of the vehicle, such as the temperature distribution (air temperature distribution) inside the vehicle. Furthermore, the temperature information acquisition device 27 can acquire temperature information about the front seats 31, 32 and the rear seat 33 near the seating position of the occupant 10, using vehicle structure information that is information specific to the vehicle 1.

[0051] The control device 21 predicts the future emotions of the occupant 10 using a trained model for predicting future emotions based on the seating position of the occupant 10, temperature information inside the vehicle, and measurement data of the occupant 10. If it is predicted that the occupant 10 will become uncomfortable in the future, the control device 21 uses the trained model for vehicle control to control the interior temperature of the vehicle by turning the air conditioner 5 on and off, changing the set temperature of the air conditioner 5, etc. The control device 21 may also control the interior temperature of the vehicle by turning on and off the seat coolers provided in the front seats 31, 32 and the rear seat 33, setting the temperature of the seat coolers, etc.

[0052] The training data set in the trained model for predicting future emotions includes multiple training data labeled with a prediction of the future emotions of the occupant 10 as output for input data such as the seating position of the occupant 10, temperature information inside the vehicle, and measurement data of the occupant 10.

[0053] The training data set in the trained model for vehicle control includes a plurality of training data labeled with output vehicle control content for input data such as the seating position of the occupant 10, temperature information inside the vehicle, and a prediction of the future emotions of the occupant 10. The vehicle control content may be, for example, temperature control inside the vehicle when it is predicted that the occupant 10 will become uncomfortable in the future based on the seating position of the occupant 10, temperature information inside the vehicle, and a prediction of the future emotions of the occupant 10.

[0054] In the in-vehicle system 2 according to the third embodiment, when it is predicted that the target occupant will become uncomfortable in the future based on temperature information about the vehicle interior, the control device 21 performs temperature control to adjust the vehicle interior temperature using the air conditioner 5. The air conditioner 5 can perform air conditioning control such as blowing air so as to prioritize and optimize the temperature at a specific location inside the vehicle, using vehicle structure information that is information specific to the vehicle 1. Furthermore, the control device 21 may adjust the vehicle interior temperature by controlling the opening and closing of an openable window provided in the vehicle 1, instead of temperature control by the air conditioner 5, as the vehicle control.

[0055] FIG. 7 is a flowchart showing an example of control performed by the control device 21 in the in-vehicle system 2 according to the third embodiment.

[0056] First, in step S21, the control device 21 identifies the seating position of the target occupant, who is the occupant 10 for whom future emotions are predicted, based on the detection results of the in-vehicle sensors. Next, in step S22, the control device 21 acquires temperature information inside the vehicle, which is spatial information inside the vehicle. Next, in step S23, the control device 21 acquires measurement data of the target occupant, such as facial expressions, from image data of the in-vehicle camera 23. Next, in step S24, the control device 21 determines whether or not a deterioration in the target occupant's emotions is predicted based on the seating position of the target occupant, the temperature information inside the vehicle, and the measurement data of the target occupant. If the control device 21 determines that a deterioration in the target occupant's emotions is not predicted, the control device 21 determines No in step S24 and terminates the series of controls without changing the in-vehicle environment. On the other hand, if the control device 21 determines that a deterioration in the target occupant's emotions is predicted, the control device 21 determines Yes in step S24 and proceeds to step S25. In step S25, the control device 21 uses a trained model for predicting future emotions to predict the future emotions of the target occupant based on the seating position of the target occupant, temperature information inside the vehicle, and measurement data of the target occupant. Next, in step S26, the control device 21 uses a trained model for vehicle control to determine the temperature control content for adjusting the temperature inside the vehicle based on the seating position of the target occupant, temperature information inside the vehicle, and the predicted future emotions of the occupant 10. Next, in step S27, the control device 21 executes temperature control based on the determined temperature control content.

[0057] As a result, in the vehicle system 2 of embodiment 3, it is possible to prevent the target occupant from becoming uncomfortable in the future due to a deterioration in the vehicle environment, such as the temperature inside the vehicle near the target occupant's seating position becoming too high or too low.

[0058] (Embodiment 4) Hereinafter, a fourth embodiment of the in-vehicle system according to the present invention will be described. Note that the description of the fourth embodiment common to the first embodiment will be omitted as appropriate.

[0059] FIG. 8 is a block diagram showing a schematic configuration of an in-vehicle system 2 according to the fourth embodiment.

[0060] In the in-vehicle system 2 according to the fourth embodiment, the spatial information about the interior of the vehicle is solar radiation information about the interior of the vehicle, and the in-vehicle system 2 is provided with a solar radiation information acquisition device 28 for acquiring the solar radiation information about the interior of the vehicle. The solar radiation information acquisition device 28 includes, for example, an illuminance sensor and acquires solar radiation information (amount of solar radiation and direction of sunlight) about the entire interior of the vehicle. The solar radiation information includes, for example, information about the amount of solar radiation and information about the direction of sunlight entering the interior of the vehicle. The solar radiation information acquisition device 28 can acquire solar radiation information about the front seats 31, 32 and the rear seat 33 near the seating position of the occupant 10 by using vehicle structure information that is information specific to the vehicle 1. The solar radiation information acquisition device 28 may also estimate and acquire current and future solar radiation information about the interior of the vehicle 1 from the direction of movement and vehicle position of the vehicle 1 using GPS information, weather information, and the like received by a car navigation system.

[0061] The control device 21 uses a trained model for predicting future emotions to predict the future emotions of the target occupant based on the seating position of the target occupant, future solar radiation information for the vehicle interior, and measurement data of the target occupant. If it is predicted that the occupant will become uncomfortable in the future, the control device 21 uses a trained model for vehicle control to perform solar radiation control by the solar radiation control device 6 as vehicle control. Examples of solar radiation control include adjusting the visible light transmittance of the window glass on the rear seat 33 side, raising and lowering the sunshade on the window glass on the rear seat 33 side, and automatically operating the sun visors on the front seats 31 and 32 side.

[0062] The training data set in the trained model for predicting future emotions includes multiple training data labeled with a prediction of the future emotions of the occupant 10 as output for input data such as the seating position of the target occupant, future solar radiation information inside the vehicle, and measurement data of the occupant 10.

[0063] The training data set in the trained model for vehicle control includes multiple pieces of training data in which the contents of vehicle control to be output are labeled for input data such as the seating position of the target occupant, future solar radiation information inside the vehicle, and predicted future emotions of the occupant 10. The contents of vehicle control include, for example, controlling solar radiation inside the vehicle by the solar radiation control device 6 when it is predicted that the occupant 10 will become uncomfortable in the future based on the seating position of the target occupant, future solar radiation information inside the vehicle, and predicted future emotions of the occupant 10.

[0064] FIG. 9 is a flowchart showing an example of control performed by the control device 21 in the in-vehicle system 2 according to the fourth embodiment.

[0065] First, in step S31, the control device 21 identifies the seating position of the target occupant, who is the occupant 10 for whom future emotions are predicted, based on the detection results of the interior sensors. Next, in step S32, the control device 21 acquires solar radiation information within the vehicle, which is spatial information within the vehicle. Next, in step S33, the control device 21 acquires measurement data of the target occupant, such as facial expressions, from image data of the interior camera 23. Next, in step S34, the control device 21 determines whether or not a deterioration in the target occupant's emotions is predicted based on the seating position of the target occupant, the solar radiation information within the vehicle, and the measurement data of the target occupant. If the control device 21 determines that a deterioration in the target occupant's emotions is not predicted, the control device 21 determines No in step S34 and terminates the series of controls without changing the interior environment of the vehicle. On the other hand, if the control device 21 determines that a deterioration in the target occupant's emotions is predicted, the control device 21 determines Yes in step S34 and proceeds to step S35. In step S35, the control device 21 uses a trained model for predicting future emotions to predict the future emotions of the occupant 10 based on the seating position of the target occupant, solar radiation information inside the vehicle, and measurement data of the target occupant. Next, in step S36, the control device 21 uses a trained model for vehicle control to determine the content of solar radiation control for adjusting the solar radiation into the vehicle interior based on the seating position of the target occupant, solar radiation information inside the vehicle, and the predicted future emotions of the target occupant. Next, in step S37, the control device 21 executes solar radiation control based on the determined content of solar radiation control.

[0066] As a result, the in-vehicle system 2 according to the fourth embodiment can prevent the target occupant from feeling uncomfortable in the future due to deterioration of the in-vehicle environment caused by future solar radiation or sunlight in the vicinity of the seating position of the target occupant.

[0067] As described above, the in-vehicle system 2 according to each embodiment can grasp the state of the space in which the occupant 10 is seated (e.g., CO2 concentration, temperature, odor, solar radiation, etc.) based on spatial information about the interior of the vehicle and the seating position of the occupant 10. The in-vehicle system 2 can also predict the future state of the interior of the vehicle based on vehicle structure information and spatial situation information. The state of the interior of the vehicle can gradually be reflected in the emotions of the occupant 10. For example, if an inappropriate temperature condition continues, the emotions of the occupant 10 will change to an uncomfortable state. Therefore, by using spatial information about the interior of the vehicle and information about the seating position of the occupant 10 as input data (explanatory variables) for machine learning, it is possible to improve the accuracy of predicting the future emotions of the occupant 10. [Explanation of symbols]

[0068] 1 vehicle 2. In-vehicle systems 4 Handle 5 Air conditioner 6. Solar radiation control device 10, 10A, 10B, 10C crew 21 Control device 22 Storage device 23 In-car camera 24 Operation Panel 25 CO2 concentration information acquisition device 26 Odor information acquisition device 27 Temperature information acquisition device 28 Solar radiation information acquisition device 31,32 Front seats 33 Back seat

Claims

1. a control device; an interior sensor for detecting the seating position of an occupant present in the vehicle; a spatial information acquisition device for acquiring spatial information inside the vehicle; a measurement data acquisition device for acquiring measurement data of the occupant; a storage device that stores a trained model for predicting future emotions of the occupant; An in-vehicle system comprising: the measurement data is facial expression of the occupant or biological information of the occupant, The control device Using the trained model, predict the future emotions of the occupant from the seating position detected by the in-vehicle sensor, the spatial information acquired by the spatial information acquisition device, and the measurement data acquired by the measurement data acquisition device; and Executing vehicle control in accordance with the predicted future emotion of the occupant. It is configured as follows: In-vehicle systems.

2. The spatial information is ２ is concentration information, the vehicle control is air conditioning control for circulating air inside the vehicle or ventilating the vehicle, The trained model is ２ The prediction result is generated by machine learning so as to derive a prediction result of the occupant's future emotion from the concentration information and the measurement data, Predicting future emotions of the occupant using the trained model includes: The seating position detected by the in-vehicle sensor and the CO ２ Concentration information and the measurement data acquired by the measurement data acquisition device are provided to the trained model, and Executing a calculation process of the trained model to obtain a prediction result of the occupant's future emotion from the trained model; It consists of The in-vehicle system according to claim 1 .

3. the spatial information is odor information, the vehicle control is air conditioning control for circulating air inside the vehicle or ventilating the vehicle, the trained model is generated by machine learning to derive a result of predicting the future emotion of the occupant from the seating position, the odor information, and the measurement data; Predicting future emotions of the occupant using the trained model includes: The seating position detected by the in-vehicle sensor, the odor information acquired by the spatial information acquisition device, and the measurement data acquired by the measurement data acquisition device are provided to the trained model; and Executing a calculation process of the trained model to obtain a prediction result of the occupant's future emotion from the trained model; It consists of The in-vehicle system according to claim 1 .

4. the spatial information is temperature information, the vehicle control is an air conditioning control for adjusting the temperature inside the vehicle, the trained model is generated by machine learning to derive a result of predicting the future emotion of the occupant from the seating position, the temperature information, and the measurement data; Predicting future emotions of the occupant using the trained model includes: The seating position detected by the in-vehicle sensor, the temperature information acquired by the spatial information acquisition device, and the measurement data acquired by the measurement data acquisition device are provided to the trained model, and Executing a calculation process of the trained model to obtain a prediction result of the occupant's future emotion from the trained model; It consists of The in-vehicle system according to claim 1 .

5. the spatial information is solar radiation information, the vehicle control is a solar radiation control for adjusting solar radiation inside the vehicle, the trained model is generated by machine learning to derive a result of predicting the future emotion of the occupant from the seating position, the solar radiation information, and the measurement data; Predicting future emotions of the occupant using the trained model includes: The seating position detected by the in-vehicle sensor, the solar radiation information acquired by the spatial information acquisition device, and the measurement data acquired by the measurement data acquisition device are provided to the trained model, and Executing a calculation process of the trained model to obtain a prediction result of the occupant's future emotion from the trained model; It consists of The in-vehicle system according to claim 1 .

Images