Information processing device, vehicle, vehicle control method and program

Abstract

The information processing device (3) according to the present disclosure includes an estimation unit (31) that estimates a user's sleep depth using sensor information acquired by a sensor (2) that detects at least one of the state of a user riding in a vehicle (1) and the environment around the user, and a control information generation unit (32) that generates control information used to control autonomous driving of the vehicle (1) using the sleep depth estimated by the estimation unit (31).

Description

[Technical field]

[0001] The present disclosure relates to an information processing device for controlling a vehicle, a vehicle, a vehicle control method, and a program. [Background technology]

[0002] A user who is in a vehicle may fall asleep while riding in the vehicle. Patent Document 1 describes a technology for preventing the sleep of a vehicle occupant from being disturbed in a series-type hybrid vehicle. The vehicle described in Patent Document 1 is capable of generating electricity with an electric motor using engine power, and prohibits power generation before and after the occupant starts to sleep. In this way, the vehicle described in Patent Document 1 prevents the occurrence of vibrations and changes in sound caused by power generation before and after the occupant starts to sleep, when the occupant's sleep is likely to be disturbed by the effects of vibrations and sound. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-131109 A Summary of the Invention [Problem to be solved by the invention]

[0004] The technology described in Patent Document 1 prevents changes in vibrations and sounds caused by power generation before and after the occupant starts to sleep, but events that disturb sleep are not limited to changes in vibrations and sounds caused by power generation; the vehicle's running itself can also affect the occupant, which is the user. For this reason, it is desirable to perform running control according to the user's sleep status.

[0005] The present disclosure has been made in consideration of the above, and has an object to provide an information processing device that can perform vehicle driving control according to a user's sleep-related state. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems and achieve the object, an information processing device according to the present disclosure uses sensor information acquired by a sensor that detects at least one of a state of a user riding in a vehicle and an environment around the user, Sleeping The vehicle includes an estimation unit that estimates a depth of sleep of a user, and a control information generation unit that generates control information used to control autonomous driving of the vehicle by using the depth of sleep estimated by the estimation unit, the estimation unit estimates the depth of sleep for a certain time ahead, and the control information generation unit determines a period during which the user is predicted to be in light sleep using the depth of sleep for the certain time ahead estimated by the estimation unit, determines a route for the vehicle so that the vehicle arrives at a parking location within the determined period, and generates control information indicating the determined route. the control information generation unit generates control information for parking the vehicle in a parking location, and when it is determined that the user is in a deep sleep after the vehicle arrives at the parking location and is parked in the parking location, using an estimation result of the sleep depth estimated by the estimation unit based on the sensor information acquired while the vehicle is parked in the parking location, generates control information indicating that the vehicle is to depart from the parking location and resume traveling. do. Effect of the Invention

[0007] The information processing device according to the present disclosure advantageously provides an advantage in that it is possible to perform vehicle driving control in accordance with the sleep-related state of the user. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing a configuration example of a vehicle according to a first embodiment; [Diagram 2] FIG. 1 shows another example of the configuration of the first embodiment. [Diagram 3] FIG. 1 is a diagram showing an example of the configuration of a processing circuit that realizes an information processing device according to a first embodiment; [Figure 4] 11 is a flowchart showing an example of a processing procedure in an information processing device according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of route setting in the second embodiment. [Figure 6] FIG. 13 is a diagram showing a configuration example of a vehicle according to a second embodiment when a user's settings are accepted; [Figure 7] 11 is a flowchart showing an example of a processing procedure in the information processing apparatus according to the second embodiment when a user setting is accepted. [Figure 8] A flowchart showing an example of a processing procedure in an information processing device according to a third embodiment. [Figure 9]FIG. 13 is a diagram showing an example of the configuration of a vehicle according to a third embodiment, which controls devices in the vehicle that affect the environment around the user. [Figure 10] A flowchart showing an example of a processing procedure in an information processing device according to a fourth embodiment. [Figure 11] FIG. 13 is a diagram showing a configuration example of a vehicle according to a fourth embodiment that estimates a wake-up time using sleep information. [Figure 12] A flowchart showing an example of a processing procedure in an information processing device according to a fourth embodiment for predicting a wake-up time using sleep information. [Figure 13] FIG. 13 is a diagram showing a configuration example of a vehicle according to a fifth embodiment. [Figure 14] FIG. 13 is a diagram showing a configuration example of an estimation unit according to a fifth embodiment when estimation is performed using machine learning. [Figure 15] Schematic diagram showing an example of a neural network DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] An information processing device, a vehicle, a vehicle control method, and a program according to embodiments will be described below in detail with reference to the drawings.

[0010] Embodiment 1 1 is a diagram showing a configuration example of a vehicle according to a first embodiment. A vehicle 1 according to the present embodiment is an autonomous vehicle, and includes a sensor 2, an information processing device 3, and a driving mechanism 4. Note that, although an example in which the vehicle 1 is an autonomous vehicle will be described here, the vehicle 1 may be capable of switching between autonomous driving and manual driving, and may be any vehicle capable of autonomous driving.

[0011] The sensor 2 is a detection device that detects at least one of the state of the user riding in the vehicle 1 and the environment around the user, and transmits the detection result as sensor information to the information processing device 3. The sensor information is information indicating at least one of the state of the user riding in the vehicle 1 and the environment around the user. The sensor 2 includes, but is not limited to, at least one of a sensor that acquires vital data of the user, a camera that captures an image of the user, a sensor that detects sound, an acceleration sensor attached to the user, an acceleration sensor that detects the acceleration of the vehicle 1, and a sensor attached to the vehicle 1. The vital data includes, but is not limited to, at least one of heart rate, body temperature, blood pressure, and respiratory rate. The sensor 2 that acquires the vital data may be worn by the user, may be a non-contact sensor fixed to the vehicle 1, or may be other than these. Note that, although one sensor 2 is illustrated in FIG. 1, the sensor 2 may be multiple, and the number of sensors 2 is not limited to the example shown in FIG. 1.

[0012] The information processing device 3 includes a sensor information acquisition unit 30, an estimation unit 31, a control information generation unit 32, a driving control unit 33, and an information storage unit 34. The sensor information acquisition unit 30 acquires sensor information by receiving sensor information from the sensor 2, and outputs the acquired sensor information to the estimation unit 31. The estimation unit 31 estimates the sleep depth of the user using the sensor information received from the sensor information acquisition unit 30, i.e., the sensor information acquired by the sensor 2. For example, the estimation unit 31 estimates whether the user is awake or asleep using the sensor information, estimates the user's sleep depth when the user is asleep, and notifies the control information generation unit 32 of the estimated sleep depth.

[0013] The sleep depth is an index showing the depth of sleep, and may be shown in two stages, for example, REM sleep, which is a light sleep, and non-REM sleep, which is a deeper sleep than REM sleep, or non-REM sleep may be further divided into three or four stages, for a total of four or five stages. In general, REM sleep and non-REM sleep appear alternately. The repetition of REM sleep and non-REM sleep is also called a sleep cycle, and the sleep cycle is, for example, about 90 minutes to 120 minutes. REM sleep is a sleep stage in which sleep is light and rapid eye movement occurs, and in REM sleep, breathing and pulse rate are high, and blood pressure and the like are higher than in non-REM sleep. Non-REM sleep is a deeper sleep than REM sleep, and as described above, non-REM sleep is also classified into three or four stages according to the depth of sleep. For example, if non-REM sleep is classified into four stages, from stage 1, which is the lightest, to stage 4, which is the deepest, then, after falling asleep, a person generally transitions from stage 1 to stage 2, stage 3, and stage 4, which are deeper stages of non-REM sleep. After stage 4 non-REM sleep, REM sleep appears. After that, REM sleep and non-REM sleep are repeated, but in the latter half of sleep, the amount of lighter stages, i.e., stages 1 and 2 non-REM sleep, increases, and REM sleep also increases.

[0014] The estimation unit 31 can estimate whether the sleep depth is one of the four stages of REM sleep or one of the three stages of non-REM sleep, based on a sleep parameter that is at least one of vital data, an index indicating body movement, and the like. The index indicating body movement is, for example, but not limited to, the number of times the user turns over in bed, the number of times a specific location moves by a certain amount or more, and the like. The index indicating body movement is, for example, but not limited to, detected by a camera that captures the user, an acceleration sensor attached to the user, a sensor (acceleration sensor, pressure sensor, etc.) attached to the vehicle 1, and the like. For example, the correspondence between the sleep depth and the numerical range of the sleep parameter is held in a table format as reference information, and the estimation unit 31 estimates the sleep depth using the sensor information and the reference information. Alternatively, the relationship between the sleep parameter and the sleep depth may be learned by machine learning, and the estimation unit 31 may estimate the sleep depth using the learning result and the acquired sleep parameter. In addition, the estimation unit 31 may further estimate the sleep depth based on the characteristics of the changes between the four or three stages of REM sleep and non-REM sleep in the above-mentioned typical human sleep (for example, a standard pattern of repetition of REM sleep and each of the four or three stages of non-REM sleep from falling asleep to waking up).

[0015] In general, the third and fourth stages of non-REM sleep are deep sleep, during which the person is not easily affected by external stimuli such as sound, vibration, and light, and even if an external stimuli occurs, the possibility of the sleep being disturbed is low, and the sleep rhythm is also less affected by the external stimuli. On the other hand, the first and second stages of non-REM sleep are light sleep, and compared to deep sleep, they are more easily affected by external stimuli, and events such as sleep being disturbed by external stimuli and sleep rhythm being disrupted are more likely to occur. Although REM sleep can be said to be a light sleep that is easy to wake up from, it is a state that is not easily affected by external stimuli, so whether REM sleep is light sleep or deep sleep can be appropriately set according to the purpose of the sleep depth. Note that the definition of deep sleep and light sleep is not limited to this example, and may be appropriately set according to the purpose of the sleep depth.

[0016] Although the sensor information indicating the environment around the user does not directly indicate the user's state, it may be used to determine whether the environment around the user is suitable for sleep. For example, the estimation unit 31 may use the sensor information indicating the environment around the user to determine whether the environment is unsuitable for sleep, and use the determination result to estimate the depth of sleep. For example, as described below, a driving mode related to sleep is set, such as a mode to promote sleep and a mode when the user is awake, and acceleration is limited in the mode to promote sleep. In this case, the estimation unit 31 may estimate that the mode is a mode to promote sleep, using sensor information acquired by the sensor 2, which is an acceleration sensor that detects the acceleration of the vehicle 1, and may indirectly grasp the state related to sleep of the user.

[0017] The information storage unit 34 stores map information and route information. The map information may be two-dimensional map information, may be three-dimensional map information, or may be a dynamic map including dynamic information. When the dynamic information of the dynamic map is received, for example, by a communication unit (not shown), the dynamic information stored in the information storage unit 34 is updated with the received dynamic information. The route information is information indicating the route of the vehicle 1, and may include the required time (or estimated arrival time) to the destination, the estimated arrival time at intermediate points, and the like.

[0018] The control information generating unit 32 generates control information used for controlling the autonomous driving of the vehicle 1 using the sleep depth estimated by the estimating unit 31, and outputs the generated control information to the driving control unit 33. The driving control of the vehicle 1 is the control of the driving itself of the vehicle 1, such as setting a travel route, controlling the speed of the vehicle 1, controlling the acceleration of the vehicle 1, controlling handling, and controlling the stopping of the vehicle 1. The control information generating unit 32 may, for example, use the sleep depth estimated by the estimating unit 31 to determine whether the user is in deep sleep or not, and generate control information to reduce external stimuli if the user is not in deep sleep.

[0019] The method of reducing external stimuli includes, but is not limited to, at least one of the following: selection of a route (driving route) with fewer external stimuli, suppression of sudden acceleration and deceleration, suppression of speed, gentle handling, and the like. In addition, for example, the estimation unit 31 may estimate the sleep depth for a certain period using the estimated sleep depth, that is, predict the sleep depth for the certain period, and notify the control information generation unit 32 of the estimation result of the sleep depth for the certain period. In this case, the control information generation unit 32 may set a route that uses the sleep depth for the certain period so that the user wakes up as naturally as possible at the point where he or she should wake up. The certain period is a period in the future, for example, a period from the present time. The point where the user should wake up may be the user's destination, or may be a place where the user plans to stop on the way.

[0020] The control information generating unit 32 may also switch the driving mode (hereinafter, also referred to as the driving mode) using the sleep depth for a certain period. For example, when the user is not asleep but awake, the driving mode may be switched to normal driving (normal driving mode) with no particular restrictions, when the user's sleep is light, the driving mode may be switched to a restricted driving mode with restrictions added so as to provide gentle driving, and when the user's sleep is deep, the driving mode may be switched to the normal driving mode. Gentle driving is driving that reduces external stimuli for the user. The definition and switching method of the driving mode are not limited to this example. In addition, when the control information generating unit 32 generates control information related to a route, the control information generating unit 32 may further generate the control information using map information stored in the information storage unit 34.

[0021] Further, the control information generating unit 32 may control devices (not shown) in the vehicle 1 by generating control information for controlling the devices in addition to the control information for controlling the driving. For example, the control information generating unit 32 may control devices in the vehicle 1, such as seats, equipment for adjusting illuminance, devices for generating sounds such as music or environmental sounds, and massage devices provided in the seats, according to the depth of sleep. For example, when the sleep is light, the seat may be controlled to be gently reclined to promote sleep of the user, or the volume of the device for generating sounds may be controlled to be lowered. Furthermore, when the time required to reach the destination is long, control for promoting sleep may be performed, such as operating a massage device or playing music for relaxation, to promote sleep before the user falls asleep. Furthermore, in order to promote sleep, the control information generating unit 32 may set the above-mentioned driving mode to a limited driving mode.

[0022] In addition, the user may be able to set whether or not to perform control to promote sleep onset. The control information generating unit 32 may, for example, set a wake-up time desired by the user, calculate backwards from the set wake-up time, determine a timing to fall asleep so that the user will be in REM sleep or light non-REM sleep at the wake-up time, and start promoting sleep before the determined timing, so that the user will be more likely to fall asleep at the determined timing or a time close to the determined timing. Alternatively, the control information generating unit 32 may set the wake-up time to the expected arrival time at the destination, determine the timing to fall asleep so that the user will be more likely to wake up around the destination, and similarly perform control to promote sleep onset.

[0023] The travel control unit 33 controls the travel of the vehicle 1 by automatic driving. The travel control unit 33 controls the travel of the vehicle 1 by controlling the travel mechanism 4 based on information acquired by sensors (not shown) such as a camera for detecting obstacles, a LiDAR (Light Detection And Ranging), and a millimeter wave sensor, map information stored in the information storage unit 34, and a self-position calculated by a self-position estimation device (not shown) for estimating a self-position. For example, when a destination is set by a user or another operator, the travel control unit 33 uses the destination and the map information to set a route to the set destination, and controls the travel mechanism 4 to travel according to the set route. The travel control unit 33 stores route information indicating the set route in the information storage unit 34. The travel control unit 33 may also calculate the required time (or estimated arrival time) to the destination and the estimated arrival time at the intermediate point, and include these in the route information. The travel control unit 33 may set a route using dynamic information on roads and traffic such as congestion information and obstacle information. In addition, the driving control for the automatic driving may be performed by any method, and since a commonly used method can be used, a detailed explanation will be omitted.

[0024] Furthermore, when the traveling control unit 33 receives control information from the control information generating unit 32, it controls the traveling mechanism 4 based on the received control information. Note that, although an example in which the traveling control unit 33 sets a route will be described here, the control information generating unit 32 may set a route and notify the traveling control unit 33 of the set route as control information, and the traveling control unit 33 may control the traveling mechanism 4 according to the route based on the control information.

[0025] The driving mechanism 4 is a mechanism for driving the vehicle 1, and includes a plurality of mechanisms for driving the vehicle 1, such as an accelerator operation device such as an accelerator pedal, a steering device, and brakes.

[0026] In this embodiment, the information processing device 3 estimates the depth of sleep of the user, and generates control information for controlling the running of the vehicle 1 based on the estimated depth of sleep. This allows the vehicle 1 to control the running of the vehicle 1 according to the state related to the user's sleep. Therefore, the vehicle 1 can make the user's environment caused by the acceleration, deceleration, handling, and route of the running of the vehicle 1 an environment suitable for the user's depth of sleep. In addition, the vehicle 1 can realize a natural awakening of the user by controlling the time when the user should wake up to be REM sleep or light non-REM sleep.

[0027] In the above-described example, the information processing device 3 is provided in the vehicle 1, but the information processing device may be provided separately from the vehicle. FIG. 2 is a diagram showing another configuration example of the present embodiment. In the example shown in FIG. 2, the vehicle system 100 includes an information processing device 3a and a vehicle 1a. Components having the same functions as those in the example shown in FIG. 1 are given the same reference numerals as those in FIG. 1, and redundant explanations are omitted. The information processing device 3a is provided separately from the vehicle 1a, and a communication unit 35 capable of communicating with the vehicle 1a is added instead of the sensor information acquisition unit 30 of the information processing device 3 shown in FIG. 1. The communication unit 35 also has a function as a sensor information acquisition unit that acquires sensor information by receiving sensor information. In addition, the driving control unit 33 in the information processing device 3 shown in FIG. 1 is provided in the vehicle 1a.

[0028] 1, the information processing device 3 is removed from the vehicle 1a, and a communication unit 36 ​​capable of communicating with the information processing device 3a is added. The communication line between the vehicle 1a and the information processing device 3a includes, for example, a wireless communication line, but may be a combination of a wireless communication line and a wired communication line. Furthermore, the communication between the vehicle 1a and the information processing device 3a may be performed via another device.

[0029] The communication unit 36 ​​of the vehicle 1a receives sensor information from the sensor 2 and transmits the received sensor information to the information processing device 3a. The communication unit 35 of the information processing device 3a receives sensor information from the vehicle 1a and outputs the received sensor information to the estimation unit 31. The operations of the estimation unit 31 and the control information generation unit 32 are similar to the example shown in FIG. 1, but the control information generation unit 32 outputs the generated control information to the communication unit 35. The communication unit 35 transmits the control information received from the control information generation unit 32 to the vehicle 1a. When the communication unit 36 ​​of the vehicle 1a receives the control information from the information processing device 3a, it outputs the received control information to the driving control unit 33. The driving control unit 33, the information storage unit 34, and the driving mechanism 4 are similar to the example shown in FIG. 1.

[0030] In addition, when the control information generation unit 32 generates control information regarding a route, the information processing device 3a may further include an information storage unit 34, and may generate the control information using map information stored in the information storage unit 34.

[0031] Next, a hardware configuration of the information processing device 3 of this embodiment will be described. In the information processing device 3 of this embodiment, a program (computer program) describing the processing in the information processing device 3 is executed on the computer system, so that the computer system functions as the information processing device 3. The computer system includes, for example, a processing circuit. FIG. 3 is a diagram showing an example of the configuration of the processing circuit that realizes the information processing device 3 of this embodiment. As shown in FIG. 3, the processing circuit includes a processor 101, a memory 102, and a communication unit 103. The processing circuit may be one circuit or multiple circuits. These are connected via a system bus. In addition, the communication unit 103 may be provided separately from the processing circuit. In addition, the computer system that realizes the information processing device 3 may further include at least one of an input unit such as a button or a keyboard that accepts input, and a display unit such as a display or a monitor.

[0032] 3, the processor 101 is a control unit such as a CPU (Central Processing Unit), and executes a program in which the processing in the information processing device 3 of this embodiment is described. The memory 102 includes various memories such as a RAM (Random Access Memory) and a ROM (Read Only Memory), and a storage device such as a hard disk, and stores the program to be executed by the processor 101, necessary data obtained in the course of processing, and the like. The memory 102 is also used as a temporary storage area for the program. The communication unit 103 is a receiver and a transmitter that executes communication processing.

[0033] Here, an example of the operation of the computer system until the program of this embodiment is in an executable state will be described. In the computer system having the above-mentioned configuration, for example, a program is installed in memory 102 from a CD-ROM or DVD-ROM set in a CD (Compact Disc)-ROM drive or DVD (Digital Versatile Disc)-ROM drive (not shown). Then, when the program is executed, the program read from memory 102 is stored in the main storage area of ​​memory 102. In this state, processor 101 executes processing as information processing device 3 of this embodiment according to the program stored in memory 102.

[0034] In the above description, a program describing the processing in the information processing device 3 is provided using a CD-ROM or DVD-ROM as a recording medium. However, this is not limited to this, and depending on the configuration of the computer system, the capacity of the program to be provided, etc., it is also possible to use a program provided over a transmission medium such as the Internet via the communication unit 103.

[0035] The program of this embodiment, for example, causes a computer system that controls vehicle 1 to execute the steps of estimating the user's sleep depth using sensor information acquired by sensor 2, and generating control information used to control the autonomous driving of vehicle 1 using the estimated sleep depth.

[0036] The estimation unit 31, the control information generation unit 32, and the driving control unit 33 shown in Fig. 1 are realized by executing a program stored in the memory 102 shown in Fig. 3 by the processor 101 shown in Fig. 3. The estimation unit 31, the control information generation unit 32, and the driving control unit 33 shown in Fig. 1 are also realized by the memory 102 shown in Fig. 3. The sensor information acquisition unit 30 shown in Fig. 1 is realized by the communication unit 103 shown in Fig. 3. The information storage unit 34 shown in Fig. 1 is a part of the memory 102 shown in Fig. 3.

[0037] Similarly, the information processing device 3a shown in Fig. 2 is also realized by a computer system including the processing circuit shown in Fig. 3. The computer system realizing the information processing device 3a may further include at least one of an input unit and a display unit in addition to the processing circuit. The communication unit 35 shown in Fig. 2 is realized by the communication unit 103 shown in Fig. 3. The information processing device 3a may be realized by a plurality of computer systems. For example, the information processing device 3a shown in Fig. 2 may be realized by a cloud system.

[0038] Embodiment 2 In the second embodiment, an example of driving control of the vehicle 1 by the information processing device 3 described in the first embodiment will be described. The configuration of the vehicle 1 in this embodiment is the same as that in the first embodiment. Note that, although the configuration in which the information processing device 3 is provided inside the vehicle 1 as shown in FIG. 1 in the first embodiment will be described as an example here, the operation of this embodiment can be similarly applied to the configuration example shown in FIG. 2 in the first embodiment. Components having the same functions as those in the first embodiment will be assigned the same reference numerals as those in the first embodiment, and duplicated explanations will be omitted.

[0039] In this embodiment, the information processing device 3 sets a route to the destination so that the user of the vehicle 1 arrives at a parking spot such as a service area (SA) and parks the vehicle, and then resumes traveling, during a time when the user is in a light sleep. This allows the information processing device 3 to prevent the user from being stimulated by the traveling of the vehicle 1 during a time when the user is in a light sleep, and promotes deep sleep for the user.

[0040] Fig. 4 is a flowchart showing an example of a processing procedure in the information processing device 3 according to the present embodiment. As shown in Fig. 4, the information processing device 3 determines whether or not the user is asleep (step S1). In the present embodiment, the sensor information acquiring unit 30 acquires sensor information from the sensor 2, for example, periodically.

[0041] In step S1, in detail, the estimation unit 31 determines whether or not the user riding in the vehicle 1 is sleeping using the sensor information received from the sensor information acquisition unit 30. The determination of whether or not the user is sleeping is performed, for example, using a camera as the sensor 2 and using image data obtained by the camera photographing the user. The estimation unit 31 may, for example, detect the movement of the user based on the image data, which is the sensor information, and determine whether or not the user is sleeping based on the movement of the user, or may determine whether or not the user's eyelids are closed based on the image data, and determine whether or not the user is sleeping based on the determination result of whether or not the eyelids are closed. Alternatively, an acceleration sensor worn by the user may be used as the sensor 2, and the estimation unit 31 may determine whether or not the user is sleeping based on sensor information, which is the detection result of the acceleration sensor. Alternatively, a sensor that acquires vital data may be used as the sensor 2, and the estimation unit 31 may determine whether or not the user is sleeping based on sensor information, which is vital data. Alternatively, the estimation unit 31 may determine whether or not the user is sleeping by combining the image data and the vital data. The method of determining whether or not the user is asleep is not limited to these examples.

[0042] If the user is not asleep (step S1 No), step S1 is repeated. If the user is asleep (step S1 Yes), the information processing device 3 estimates the sleep depth to the destination (step S2). In detail, the estimation unit 31 extracts the route and the required time to the destination from the route information stored in the information storage unit 34, estimates the sleep depth at each time from the present to the time of arrival at the destination using the extracted information, and outputs the estimated sleep depth to the control information generation unit 32. Note that, as described in the first embodiment, the route and the required time to the destination are calculated by, for example, the driving control unit 33 and stored in the information storage unit 34 as route information. Also, as described in the first embodiment, the control information generation unit 32 may set a route using the destination and map information and obtain the required time, and in this case, the control information generation unit 32 stores the route information in the information storage unit 34. Note that, here, an example in which the estimation unit 31 estimates the sleep depth up to the destination will be described, but the present invention is not limited to this, and the estimation unit 31 may estimate the sleep depth up to a certain time ahead. As described above, the certain time may be the time required to reach the destination, or may be a time shorter than the time required to reach the destination, such as half the time required to reach the destination, or may be a predetermined time.

[0043] Next, the information processing device 3 sets a route to arrive at the parking location during a light sleep period (step S3). In detail, the control information generating unit 32 determines a period during which the user is predicted to be in light sleep using the depth of sleep to the destination estimated by the estimating unit 31, determines a route for the vehicle 1 so as to arrive at the parking location within the determined period, and generates control information indicating the determined route. This control information may be output to the driving control unit 33, or may be stored in the information storage unit 34 as route information.

[0044] For example, the control information generating unit 32 first calculates the position of the vehicle 1 during the light sleep time period by using the route to the destination and the sleep depth to the destination received from the estimation unit 31. If there is a parking spot in the vicinity of the calculated position, the control information generating unit 32 determines a route to arrive at the parking spot during that time period and park at the parking spot. The parking spot includes, for example, at least one of SA and PA (parking area), but may also include a rest area such as a roadside station, or a parking lot where parking is permitted. The parking lot may be a parking lot for a store, a park, or the like. If there is a parking spot on the initially set route, the driving speed may be determined so that the parking spot is arrived at during the light sleep time period, and the determined driving speed may also be output to the driving control unit 33 as control information. If there is no parking spot on the initially set route, the control information generating unit 32 selects a parking spot that is to be arrived at during the light sleep time period from among parking spots around the route, and sets a route that passes through the parking spot. That is, the control information generating unit 32 updates the route information stored in the information storage unit 34 to route information (control information) indicating a route passing through the parking location. Alternatively, the control information generating unit 32 may output the route information indicating the route passing through the parking location to the driving control unit 33 as control information. Furthermore, the control information generating unit 32 generates control information for parking the vehicle 1 at the parking location, and outputs the generated control information to the driving control unit 33. Note that an instruction to park the vehicle 1 at the parking location may be included in the route information. As a result, the driving control unit 33 controls the driving of the vehicle 1 based on the updated route information, and when the vehicle arrives at the parking location, controls the driving mechanism 4 to park the vehicle 1 at the parking location.

[0045] In addition, when it is estimated that light sleep periods will occur a predetermined number of times or more before arriving at the destination, it is possible to determine parking locations corresponding to two or more of the light sleep periods, assuming that the vehicle will park in a parking location for a certain period of time before resuming driving, and set a route that passes through the determined two or more parking locations. For example, when the required time to the destination is seven hours or more, four to five sleep cycles will occur, and the latter sleep cycles are preparations for awakening, so the control information generating unit 32 may set a route that passes through two parking locations so as to promote deep sleep in the first two sleep cycles.

[0046] Next, the information processing device 3 determines whether the vehicle 1 has arrived at the parking spot (step S4). In detail, the control information generating unit 32 may determine that the vehicle 1 has arrived at the parking spot by receiving a notification from the driving control unit 33 that the vehicle 1 has been parked in the parking spot, or may obtain the current position of the vehicle 1 from the driving control unit 33 and determine that the vehicle 1 has arrived at the parking spot if the current position is within the parking spot.

[0047] If the user has not arrived at the parking location (step S4: No), step S4 is repeated. If the user has arrived at the parking location (step S4: Yes), the information processing device 3 estimates the sleep depth (step S5). In detail, the control information generating unit 32 instructs the estimating unit 31 to estimate the sleep depth, and the estimating unit 31 estimates the current sleep depth of the user using the sensor information and outputs the estimation result to the control information generating unit 32.

[0048] The information processing device 3 determines whether the user is in deep sleep or not (step S6). In detail, the control information generating unit 32 determines whether the user is in deep sleep or not based on the estimation result received from the estimation unit 31.

[0049] If the user is not in deep sleep (step S6 No), the process from step S5 is repeated. If the user is in deep sleep (step S6 Yes), the information processing device 3 resumes driving (step S7). In detail, the control information generating unit 32 generates control information instructing to resume driving, and outputs the generated control information to the driving control unit 33. The driving control unit 33 controls the driving mechanism 4 to resume driving of the vehicle 1 based on the control information received from the control information generating unit 32. In this way, when the control information generating unit 32 determines that the user is in deep sleep after the vehicle 1 arrives at the parking location, using the estimation result of the sleep depth estimated by the estimation unit 31 based on the sensor information, the control information generating unit 32 generates control information indicating that the vehicle 1 is to depart from the parking location and resume driving.

[0050] After resuming traveling, the information processing device 3 judges whether parking is unnecessary or not (step S8). In detail, the control information generating unit 32 judges that parking is unnecessary when parking has already been completed in all parking locations along the route indicated by the route information. Note that even if there are parking locations that have not yet been parked among all parking locations along the route indicated by the route information, the control information generating unit 32 may judge that parking is unnecessary in step S8 even if there are parking locations that have not yet been parked, if traveling takes longer than originally planned and arrival at the destination is expected to be delayed by a specified time or more.

[0051] If parking is not necessary (step S8 No), the process from step S4 is repeated. If parking is not necessary (step S8 Yes), the information processing device 3 drives the vehicle 1 to the destination (step S9). In detail, in step S9, if the vehicle has already been parked in all the parking locations to be passed through, the driving control unit 33 controls the driving mechanism 4 to drive the vehicle 1 to the destination based on the route information. If there is a parking location that has not yet been parked but it is determined in step S8 that parking is not necessary, in step S9, the control information generating unit 32 sets a route that does not pass through the remaining parking locations (parking locations that have not yet been reached) that were to be passed through. That is, the control information generating unit 32 updates the route information stored in the information storage unit 34 to route information indicating a route that does not pass through the remaining parking locations. As a result, the driving control unit 33 controls the driving of the vehicle 1 based on the updated route information to drive the vehicle 1 to the destination.

[0052] 4, the information processing device 3 resumes traveling after the user sleeps deeply after the vehicle 1 is parked in a parking place, but the present invention is not limited to this and may resume traveling after the vehicle is parked in a parking place for a set time. The information processing device 3 may resume traveling when at least one of the conditions of the user sleeping deeply and the vehicle being parked in a parking place for a set time is satisfied.

[0053] FIG. 5 is a conceptual diagram showing an example of route setting in this embodiment. In the example shown in FIG. 5, a route 201 from a departure point to a destination is set. It is assumed that the user falls asleep at a point 202 on the route 201 after the vehicle 1 leaves the departure point. In the example shown in FIG. 5, the information processing device 3 predicts the depth of sleep of the user, determines that the user is in light sleep during the time (period) when the user is traveling in the section 203, and sets the route 201 to pass through SA301 and SA302. The section 204 is a section corresponding to the time (period) when the user is in deep sleep. In this way, the vehicle 1 is parked at parking places such as SA301 and 302 when the user is in light sleep, so that it is possible to prevent the user from being stimulated by the traveling of the vehicle 1 and promote the user's deep sleep. Note that FIG. 5 is an example, and the number of routes and parking places to be set is not limited to the example shown in FIG. 5. In addition, the parking places are not limited to SAs, and may be PAs as described above, or parking lots of stores, parks, etc.

[0054] In the above example, the user is encouraged to sleep deeply by parking the vehicle in a parking space, but the user may wish to take a restroom break, go shopping, or get off the vehicle at the parking space. For this reason, the user may be allowed to set whether to encourage deep sleep or to wake up the user at the parking space.

[0055] Fig. 6 is a diagram showing an example of the configuration of a vehicle according to the present embodiment when accepting user settings. The vehicle 1b shown in Fig. 6 is similar to the vehicle 1 shown in Fig. 1, except that an awakening promotion device 5 is added and an information processing device 3b is provided instead of the information processing device 3. Components having the same functions as those in the example shown in Fig. 1 are given the same reference numerals as those in Fig. 1, and duplicated explanations will be omitted.

[0056] As shown in FIG. 6, the information processing device 3b is obtained by adding an input receiving unit 37 to the information processing device 3 shown in FIG. 1. The input receiving unit 37 receives an input of a selection result indicating whether the user will get off at the parking spot or not. The input receiving unit 37 outputs the received selection result to the control information generating unit 32. If the selection result indicates that the user will get off at the parking spot, the control information generating unit 32 sets the mode of the operation at the parking spot to awakening promotion. If the mode of the operation at the parking spot is set to awakening promotion, the control information generating unit 32 generates control information instructing the user to awaken when the user arrives at the parking spot, and outputs the generated control information to the awakening promotion device 5. When the awakening promotion device 5 receives the control information from the control information generating unit 32, it performs an operation to promote the awakening of the user. The operation to promote the awakening of the user may be, for example, an operation to generate a sound such as a voice, an alarm sound, or music that notifies the user that the user has arrived at the parking spot, or an operation to vibrate the seat, or may be other than these. If the selection result indicates that the user will not get off at the parking location, the control information generation unit 32 sets the mode of operation at the parking location to sleep promotion, and does not perform control to wake the user up even when the user arrives at the parking location, as in the example shown in Figure 4.

[0057] Fig. 7 is a flowchart showing an example of a processing procedure in the information processing device 3b of this embodiment when accepting a user's setting. Steps S1 to S4 are the same as the example shown in Fig. 4. If the answer is Yes in step S4, the information processing device 3b judges whether or not the setting is for awakening promotion (step S11). In detail, the control information generating unit 32 judges whether or not the mode of operation at the parking location is for awakening promotion. If the setting is not for awakening promotion (step S11 No), steps S5 to S9 are performed as in Fig. 4.

[0058] If the setting is for awakening promotion (Yes in step S11), the information processing device 3b promotes the awakening of the user (step S12). In detail, the control information generating unit 32 generates control information for instructing the user to be awakened, and outputs the generated control information to the awakening promotion device 5. As a result, when the awakening promotion device 5 receives the control information from the control information generating unit 32, it performs an operation for promoting the awakening of the user.

[0059] Next, the information processing device 3b determines whether or not a departure instruction has been given, i.e., an instruction to depart vehicle 1b has been given (step S13), and if a departure instruction has been given (step S13 Yes), the process proceeds to step S7. If a departure instruction has not been given (step S13 No), the information processing device 3b repeats step S13. For example, an instruction to depart vehicle 1b is input by a user. When the input receiving unit 37 receives an input from the user instructing vehicle 1b to depart, it outputs information indicating that vehicle 1b has been instructed to depart to the control information generating unit 32. In step S13, the control information generating unit 32 determines whether or not a departure instruction has been given based on whether or not information indicating that vehicle 1b has been instructed to depart has been received from the input receiving unit 37.

[0060] In the example shown in FIG. 7, when the operation mode is set to sleep promotion, the same process as in FIG. 4 is performed, and when the operation mode is set to awakening promotion, the user is awakened when the user arrives at the parking spot. Even when the user's awakening is promoted at the parking spot, the route is set so that the user arrives at the parking spot during a light sleep time, so that the user is likely to awaken naturally. In the example shown in FIG. 7, the user can set the operation mode, so that the information processing device 3b can perform an operation according to the user's request. Note that an input receiving unit 37 and an awakening promotion device 5 are added to the vehicle 1a shown in FIG. 2 of the first embodiment, and information received by the input receiving unit 37 is transmitted to the information processing device 3a via the communication unit 36, so that the information processing device 3a may perform a process similar to the process shown in FIG. 7.

[0061] In the examples shown in Fig. 6 and Fig. 7, the vehicle 1b is provided with the awakening promotion device 5, but the vehicle 1b may not be provided with the awakening promotion device 5. Since the user may sense the arrival of the vehicle at the parking spot when the user is in a light sleep state and wake up naturally, the information processing device 3b may wait for the user to get off the vehicle for a certain period of time after arriving at the parking spot. If the user does not wake up after the certain period of time, the information processing device 3b may perform an operation in which the operation mode is set to sleep promotion.

[0062] The information processing device 3b shown in FIG. 6 is realized by, for example, the processing circuit exemplified in FIG. 3 and an input unit (not shown), and the input receiving unit 37 is realized by the input unit.

[0063] As described above, in this embodiment, the route of the vehicle 1, 1b is set so that the vehicle 1, 1b arrives at the parking spot when the user is in a light sleep state, and when the vehicle 1, 1b arrives at the parking spot, the vehicle 1, 1b stops at the parking spot. This allows the vehicle 1, 1b to avoid being disturbed by the stimulation caused by the vehicle 1, 1b's running during the user's light sleep state, and promotes deep sleep for the user. In other words, running control can be performed according to the user's sleep state.

[0064] Embodiment 3 In the third embodiment, as in the second embodiment, an example of driving control of the vehicle 1 by the information processing device 3 described in the first embodiment will be described. The configuration of the vehicle 1 in this embodiment is the same as that in the first embodiment. Note that, although the configuration in which the information processing device 3 is provided inside the vehicle 1 as shown in FIG. 1 in the first embodiment will be described as an example here, the operation of this embodiment can be similarly applied to the configuration example shown in FIG. 2 in the first embodiment. Components having the same functions as those in the first embodiment are given the same reference numerals as those in the first embodiment, and duplicated explanations will be omitted.

[0065] Fig. 8 is a flowchart showing an example of a processing procedure in the information processing device 3 of the present embodiment. As shown in Fig. 8, the information processing device 3 determines whether or not the user is asleep, similarly to step S1 of the second embodiment. Note that also in the present embodiment, the sensor information acquiring unit 30 acquires sensor information from the sensor 2, for example, periodically.

[0066] If the user is not asleep (step S1 No), the information processing device 3 determines that the vehicle 1 is to be driven in normal driving (step S21), and step S1 is repeated. For example, as described in the first embodiment, the user can set the driving mode of the vehicle 1 to two types of modes, the normal driving mode and the limited driving mode, and the control information generating unit 32 sets the driving mode to the normal driving mode. Note that if the driving mode is already set to the normal driving mode, there is no need to change the driving mode, and the control information generating unit 32 keeps the driving mode in the normal driving mode.

[0067] If the user is asleep (Yes in step S1), the information processing device 3 estimates the sleep depth (step S22). In detail, the estimation unit 31 estimates the user's current sleep depth using the sensor information, and outputs the estimation result to the control information generation unit 32. Alternatively, the estimation unit 31 may predict the sleep depth for a certain time ahead using the estimation result of the user's current sleep depth, and output the prediction result to the control information generation unit 32.

[0068] Next, the information processing device 3 judges whether the user is in light sleep (step S23). In detail, the control information generating unit 32 judges whether the user is in light sleep based on the estimation result received from the estimation unit 31. If the user is in light sleep (step S23 Yes), the information processing device 3 decides to set the vehicle 1 to the restricted driving mode (step S24), and repeats the process from step S22. In detail, in step S24, the control information generating unit 32 generates control information for making the vehicle 1 drive more gentle than in normal driving without restrictions. For example, the control information generating unit 32 sets the driving mode to the restricted driving mode. Note that, if the driving mode is already set to the restricted driving mode, the driving mode does not need to be changed, and the control information generating unit 32 generates control information instructing to keep the driving mode in the restricted driving mode, and outputs the generated control information to the driving control unit 33. In the restricted driving mode, as described in the first embodiment, driving control is performed to make the driving gentle.

[0069] Specifically, in the limited driving mode, the speed and / or acceleration may be limited by being set to a threshold or less, or the driving route may be set to a route that allows for gentle driving. Examples of routes that allow for gentle driving include, but are not limited to, routes with few sharp curves, routes with few unevenness on the road surface, and routes with gentle gradients. For example, the map information includes values ​​indicating the curvature of the curve, the unevenness of the road surface, and the gradient of the road. The control information generating unit 32 can determine that a curve is a sharp curve when the curvature of the curve is equal to or greater than a threshold, determine that a route with few unevenness is a route with a numerical value indicating the unevenness of the road surface equal to or less than a threshold, and determine that a route with a gentle gradient is a route with a gentle gradient when the numerical value indicating the gradient of the road is equal to or less than a threshold. As described above, the control information generating unit 32 can receive a predicted value of the sleep depth for a certain time ahead from the estimating unit 31, and in this case, set the driving mode for a period in which the predicted result indicates light sleep to the limited driving mode.

[0070] If the user is not in a light sleep (No in step S23), the information processing device 3 determines that the vehicle 1 is to be driven in normal driving (step S25), and repeats the process from step S22. In detail, in step S25, for example, the control information generating unit 32 sets the driving mode to the normal driving mode. Note that if the driving mode is already set to the normal driving mode, there is no need to change the driving mode, and the control information generating unit 32 keeps the driving mode in the normal driving mode.

[0071] By performing the process illustrated in FIG. 8, the information processing device 3 can control the vehicle 1 to run gently during the time when the user is in a light sleep. That is, the information processing device 3 can control the vehicle 1 to run gently during the time when the user is in a light sleep, so as not to disturb the user's sleep. In addition, in the above-mentioned example, the vehicle 1 runs gently during the time when the user is in a light sleep, so as not to disturb the user's sleep. In addition, the information processing device 3 can suppress external stimuli to the user caused by factors other than running during the time when the user is in a light sleep. For example, the external stimuli can be suppressed by adjusting at least one of the brightness of the surroundings and the sound of the surroundings as the environment around the user. For example, in the above-mentioned step S24, the control information generating unit 32 can set a route that avoids bright places, or a route that avoids noisy places. The route that avoids bright places is, for example, but is not limited to, a route that avoids areas with many stores when the vehicle 1 runs at night. The route that avoids noisy places is, for example, but is not limited to, a route that avoids the vicinity of a construction site, or a route that avoids areas with many people or cars.

[0072] Furthermore, the control information generator 32 may suppress external stimuli to the user by controlling devices in the vehicle 1 that affect the environment around the user. Fig. 9 is a diagram showing an example of the configuration of a vehicle according to this embodiment that controls devices in a vehicle 1c that affect the environment around the user. The vehicle 1c shown in Fig. 9 is obtained by adding a speaker 11, a sound reduction device 12, and a dimmer device 13 to the vehicle 1 according to the first embodiment.

[0073] The speaker 11 outputs the audio data output from an audio data output device (not shown) as sound. The audio data output device is, for example, a device that plays music in the vehicle 1c, a device that acquires audio data to be broadcasted or distributed, etc. The sound reduction device 12 is a device that reduces the sound in the vehicle 1c, and may be a combination of a sound absorbing sheet and a device that controls the insertion and removal of the sound absorbing sheet, or may be a device that detects noise in the vehicle 1c and performs noise canceling by generating a sound in phase opposite to the detected noise to cancel the noise through electrical processing, or may be other than these. The light reduction device 13 is a device that reduces the light in the vehicle 1c, and may be, for example, a light control film that is provided on a rear window of the vehicle 1c and is capable of adjusting the light transmittance, or may be a curtain that blocks light while the vehicle 1c is parked, or may be other than these.

[0074] In the example shown in FIG. 9, the control information generating unit 32 further generates control information for operating the speaker 11, the sound reduction device 12, and the dimmer device 13 in step S24 shown in FIG. 8, and outputs the generated control information to the speaker 11, the sound reduction device 12, and the dimmer device 13, thereby suppressing sound and light in the vehicle 1. For example, the control information generating unit 32 generates control information for lowering the volume of the speaker 11 or turning off the speaker 11, generates control information for causing the sound reduction device 12 to suppress sound in the vehicle 1, and generates control information for causing the dimmer device 13 to suppress light in the vehicle 1. This makes it possible to suppress not only stimulation caused by driving, but also stimulation caused by the surrounding environment other than driving, for the user. In the example shown in FIG. 9, an example is shown in which the vehicle 1c is equipped with the speaker 11, the sound reduction device 12, and the dimmer device 13, but this is not limited thereto, and the vehicle 1c may be equipped with one or two of the speaker 11, the sound reduction device 12, and the dimmer device 13.

[0075] Note that a speaker 11, a sound reduction device 12, and a dimmer device 13 may be added to the vehicle 1a shown in Fig. 2 of the first embodiment, and the information processing device 3a may generate control information for controlling these devices and transmit the control information to the vehicle 1a via the communication unit 35. In this case, the vehicle 1a may also include one or two of the speaker 11, the sound reduction device 12, and the dimmer device 13.

[0076] In addition, the information processing device 3 may perform both the operation of embodiment 2 and the operation described in this embodiment, and the vehicle 1a and information processing device 3a shown in Figure 2 may perform both the operation of embodiment 2 and the operation described in this embodiment.

[0077] Further, although an example has been described here in which both stimuli caused by driving and stimuli caused by the user's surrounding environment are suppressed, the vehicle 1c may be configured to suppress only stimuli caused by the user's surrounding environment when the user is in a light sleep.

[0078] Embodiment 4 In the fourth embodiment, as in the second and third embodiments, an example of driving control of the vehicle 1 by the information processing device 3 described in the first embodiment will be described. The configuration of the vehicle 1 in this embodiment is the same as that in the first embodiment. Note that, although the configuration in which the information processing device 3 is provided inside the vehicle 1 as shown in FIG. 1 in the first embodiment will be described as an example here, the operation of this embodiment can be similarly applied to the configuration example shown in FIG. 2 in the first embodiment. Components having the same functions as those in the first embodiment will be assigned the same reference numerals as those in the first embodiment, and duplicated explanations will be omitted.

[0079] FIG. 10 is a flowchart showing an example of a processing procedure in the information processing device 3 of this embodiment. Steps S1 and S2 are the same as steps S1 and S2 of the second embodiment. After step S2, the information processing device 3 predicts the wake-up time (step S31). In detail, the estimation unit 31 predicts the wake-up time of the user using the estimated sleep depth. For example, the estimation unit 31 predicts the wake-up time using the estimation result of the sleep depth and the standard sleep pattern of a person described in the first embodiment, and outputs the prediction result to the control information generation unit 32. In detail, for example, the estimation unit 31 estimates the time of sleep onset and the sleep cycle using the estimation result of the sleep depth, calculates a time period in which REM sleep occurs within a certain period including the expected arrival time at the destination, and sets the time within the calculated time period as the expected wake-up time. For example, the control information generation unit 32 may set the time in the center of the time period in which REM sleep occurs as the expected wake-up time.

[0080] Next, the information processing device 3 sets a route that will arrive at the destination within a certain time from the predicted wake-up time (step S32). In detail, the control information generating unit 32 determines a route to the destination of the vehicle 1 so that the vehicle 1 arrives at the destination within a certain time from the wake-up time predicted in step S31, generates route information that is control information indicating the determined route, and stores the route information in the information storage unit 34. Alternatively, the control information generating unit 32 may output the control information indicating the determined route to the driving control unit 33. The vehicle 1 can arrive at the destination within a certain time from the predicted wake-up time by driving based on this route information. This can increase the possibility that the user will naturally wake up near the destination. In this way, even in this embodiment, driving control can be performed according to the user's sleep-related state.

[0081] In the above example, the wake-up time is predicted using the sleep depth, but this is not limited to the above. The wake-up time may be set by the user, or may be estimated based on sleep information indicating at least one of the user's sleep history, the user's wake-up time history, and the user's planned wake-up time, as described below.

[0082] FIG. 11 is a diagram showing a configuration example of a vehicle according to the present embodiment that estimates a wake-up time using sleep information. As shown in FIG. 11, the vehicle 1d includes an information processing device 3d instead of the information processing device 3. The information processing device 3d includes a sleep information acquisition unit 38 added to the information processing device 3. The sleep information acquisition unit 38 acquires sleep information by receiving sleep information from an information providing device 6 that stores the user's sleep information, and outputs the acquired sleep information to the estimation unit 31. Note that FIG. 11 shows an example in which the information providing device 6 is a mobile terminal such as a smartphone, tablet, or personal computer carried by the user, and the information providing device 6 is present inside the vehicle 1d, but the information providing device 6 may be present outside the vehicle 1d. In addition, the information providing device 6 is not limited to a mobile terminal carried by the user, and may be a server device that manages sleep information of multiple users.

[0083] The sleep information is, for example, information managed by a sleep app (application software) that acquires and manages information related to the user's sleep, but is not limited to this. For example, the information providing device 6 may have an alarm function, and the wake-up time set in the alarm function may be used as the sleep information. In addition, if the information providing device 6 has a schedule management function, and the user registers the wake-up time in the schedule, the registered wake-up time may be used as the sleep information.

[0084] The sleep information managed by the sleep app includes, for example, the user's sleep onset time, wake-up time, and sleep cycle. When predicting the wake-up time using the sleep information managed by the sleep app, the estimation unit 31 may, for example, calculate the average value of the wake-up time using the history of the user's wake-up time, and use the calculated average value as the predicted value of the wake-up time. Alternatively, the estimation unit 31 may calculate the average value of the sleep time using the history of the sleep onset time and the wake-up time, estimate the user's sleep onset using sensor information, and use the time that is the calculated average value from the estimated sleep onset time as the predicted value of the wake-up time.

[0085] In addition, when the sleep information is the wake-up time set in the alarm function, the estimation unit 31 may use the wake-up time indicated by the sleep information as the predicted value of the wake-up time. In addition, when the sleep information is the wake-up time registered in the schedule, the estimation unit 31 may use the wake-up time indicated by the sleep information as the predicted value of the wake-up time. In addition, when a schedule for the target day of the wake-up time prediction is registered, the wake-up time in the schedule for the target day of the wake-up time prediction is used as the predicted value of the wake-up time, and when a schedule for the target day of the wake-up time prediction is not registered, the wake-up time registered on another day of the same day of the week may be used as the predicted value of the wake-up time, or the most common wake-up time among the registered wake-up times may be used as the predicted value of the wake-up time.

[0086] The information processing device 3d shown in Fig. 11 is realized by, for example, the processing circuit exemplified in Fig. 3. The sleep information acquisition unit 38 is realized by the communication unit 103.

[0087] Fig. 12 is a flowchart showing an example of a processing procedure in the information processing device 3d of this embodiment that predicts the wake-up time using sleep information. Step S1 is the same as the example shown in Fig. 10. If the answer is Yes in step S1, the information processing device 3d predicts the wake-up time using the sleep information (step S41). In detail, the estimation unit 31 predicts the wake-up time of the user using the sleep information received from the sleep information acquisition unit 38, and outputs the prediction result to the control information generation unit 32. Step S32 is the same as the example shown in Fig. 10.

[0088] Note that a sleep information acquiring unit 38 may be added to the vehicle 1a or the information processing device 3a shown in FIG. 2 of the first embodiment, and the information processing device 3a may predict the wake-up time of the user based on the sleep information in the same manner as in the present embodiment, and set a route to arrive at the destination within a certain time from the predicted wake-up time.

[0089] In addition, the information processing device 3d may perform at least one of the operations of embodiment 2 and embodiment 3 and the operations described in this embodiment, and the vehicle 1a and information processing device 3a shown in FIG. 2 may perform at least one of the operations of embodiment 2 and embodiment 3.

[0090] As described above, in this embodiment, the wake-up time is predicted and a route is set so that the user arrives at the destination within a certain time from the predicted wake-up time, which increases the possibility that the user will wake up naturally near the destination, thereby improving the user's comfort.

[0091] Embodiment 5. 13 is a diagram showing a configuration example of a vehicle 1e according to a fifth embodiment. The vehicle 1e according to the present embodiment is similar to the vehicle 1 according to the first embodiment, except that the vehicle 1e includes an information processing device 3e instead of the information processing device 3. Components having the same functions as those in the first embodiment are given the same reference numerals as those in the first embodiment, and duplicated explanations will be omitted.

[0092] 13, the information processing device 3e is obtained by adding an identification unit 39 and a characteristic information storage unit 40 to the information processing device 3 of the first embodiment. Note that, although the information processing device 3e is described here by way of example in which it is provided inside the vehicle 1e as shown in FIG. 1 of the first embodiment, the identification unit 39 and the characteristic information storage unit 40 may be added to the information processing device 3a in the configuration example shown in FIG. 2 of the first embodiment, and the operation of this embodiment may be performed.

[0093] In this embodiment, the sensor 2 includes, for example, a camera, and the sensor information includes image data acquired by capturing an image of a user with the camera. The sensor information acquisition unit 30 outputs the image data to the identification unit 39. The identification unit 39 identifies the user using the image data, and outputs an identification result (user identification result) to the estimation unit 31. The user identification may be an identification of the individual user, or an identification of an attribute of the user. The attribute includes, for example, at least one of gender and age, but is not limited thereto.

[0094] The user is identified, for example, by using face recognition technology. When identifying an individual user, image data showing an image of the user's face is registered in advance, and the identification unit 39 identifies the individual user by comparing the registered image data with the photographed data. The age does not have to be the age itself, and may be indicated by classifications such as infant, child, and adult, or may be indicated by classifications such as age group, such as under 10 years old, teens, and twenties. The age classifications are not limited to these examples. In the example shown in FIG. 13, the identification unit 39 identifies the user based on the photographed data, but may identify the user using vital data acquired as sensor information, or may identify the user by combining the photographed data and vital data.

[0095] The characteristic information storage unit 40 stores characteristic information indicating characteristics related to sleep for each user or for each attribute. The characteristic information includes at least one of information indicating, for example, the user's sleep cycle, sleep time, wake-up time, characteristics of vital data for each sleep depth, characteristics of the user's body movement for each sleep depth, characteristics of vital data of body movement during sleep and wakefulness, characteristics of body movement during sleep and wakefulness, and the like, and is used for estimation in the estimation unit 31. The characteristic information may be information in a table format in which data of each item is stored for each user, or may be a learned model learned by machine learning as described later. When the characteristic information is information in a table format, the characteristic information is calculated by statistically processing data accumulated in advance for each user or for each attribute. The characteristic information may also be calculated based on sensor information acquired by the sensor 2 when the user previously got into the vehicle 1e.

[0096] The estimation unit 31 estimates an index related to sleep using the identification result received from the identification unit 39 and the characteristic information stored in the characteristic information storage unit 40. The index related to sleep may be a sleep depth, a wake-up time, or a classification of whether or not the user is asleep, or may be two or more of these. That is, in the determination of step S1 shown in FIG. 4, FIG. 7, FIG. 8, FIG. 10, and FIG. 12, the estimation unit 31 may estimate whether or not the user is asleep using the identification result received from the identification unit 39 and the characteristic information stored in the characteristic information storage unit 40. In estimating whether or not the user is asleep, at least one of information indicating, for example, characteristics of vital data of body movements during sleep and wakefulness, characteristics of body movements during sleep and wakefulness, etc. is used as the characteristic information.

[0097] In addition, the estimation unit 31 may estimate the sleep depth using the identification result received from the identification unit 39 and the characteristic information stored in the characteristic information storage unit 40 in estimating the sleep depth in FIG. 4, FIG. 7, FIG. 8, and FIG. 10. In estimating the user's sleep depth, at least one of information indicating the user's sleep cycle, vital data characteristics for each sleep depth, and body movement characteristics of the user for each sleep depth is used as the characteristic information. In addition, the estimation unit 31 may estimate the sleep depth using the identification result received from the identification unit 39 and the characteristic information stored in the characteristic information storage unit 40 when estimating the wake-up time described in the fourth embodiment. In estimating the user's wake-up time, at least one of information indicating the sleep time, wake-up time, and the like is used as the characteristic information. In this case, the characteristic information may be an actual value of at least one of the user's wake-up time and sleep time.

[0098] As described above, the characteristic information may be a trained model. FIG. 14 is a diagram showing a configuration example of the estimation unit 31 of this embodiment when estimation is performed using machine learning. In the example shown in FIG. 14, the estimation unit 31 includes a learning unit 311 and an inference unit 312. The learning unit 311 generates a trained model by supervised learning using a plurality of data sets, for example, each of which is composed of input data that is a feature amount and a sleep-related index that is corresponding ground truth data, and stores the generated trained model in the characteristic information storage unit 40 as characteristic information. For example, the feature amount includes information calculated from sensor information.

[0099] When the trained model is a model for inferring whether or not the user is asleep, the input data includes at least one of information indicating the characteristics of vital data of body movements during sleep and wakefulness, characteristics of body movements during sleep and wakefulness, and the like. When the trained model is a model for inferring sleep depth, the input data includes at least one of information indicating the user's sleep cycle, characteristics of vital data for each sleep depth, and characteristics of the user's body movements for each sleep depth, and the like. The input data may be calculated from data previously acquired by the sensor 2, may be calculated from data acquired by a device other than the sensor 2, or may be a mixture of these. The input data may also be acquired by a sleep app or the like. The correct answer data may be determined, for example, by acquiring brain waves separately and using the brain waves, may be determined based on the result of giving a stimulus to the user, or may be determined by a method other than these.

[0100] Any algorithm may be used for supervised learning to generate a trained model in the learning unit 311, and for example, a neural network model may be used. The neural network is composed of an input layer made up of a plurality of neurons, an intermediate layer (hidden layer) made up of a plurality of neurons, and an output layer made up of a plurality of neurons. The intermediate layer may be one layer, or two or more layers.

[0101] Fig. 15 is a schematic diagram showing an example of a neural network. For example, in the case of a three-layered neural network as shown in Fig. 15, when multiple inputs are input to the input layer (X1-X3), the values ​​are multiplied by weight W1 (w11-w16) and input to the intermediate layer (Y1-Y2), and the result is further multiplied by weight W2 (w21-w26) and output from the output layer (Z1-Z3). This output result changes depending on the value of weight W1 and the value of weight W2.

[0102] In this embodiment, the weights W1 and W2 are adjusted so that the output from the output layer when the feature is input approaches the index related to sleep, which is the correct data, to learn the relationship between the feature and the correct data. Note that the machine learning algorithm is not limited to a neural network, and may be other algorithms such as a support vector machine. Also, the machine learning used to generate the trained model in the learning unit 311 is not limited to supervised learning, and may be reinforcement learning, etc.

[0103] The inference unit 312 calculates features using the sensor information received from the sensor information acquisition unit 30, and infers a sleep-related index by inputting the features into the trained model stored in the characteristic information storage unit 40. In this way, the sleep-related index is estimated.

[0104] The trained model may be generated for each individual user or for each attribute. For example, a trained model may be generated for each age group. When the identification result is an identification result of an individual user, the estimation unit 31 infers a sleep-related index using a trained model corresponding to the individual user. When the identification result is an identification result of an attribute of the user, the estimation unit 31 infers a sleep-related index using a trained model corresponding to the attribute. Alternatively, instead of generating a trained model for each individual user or attribute, the identification information or attributes of the individual user may also be input as a feature to generate a trained model, and the identification information or attributes of the individual user may also be input as a feature to the trained model during inference.

[0105] In addition, in FIG. 14, an example is shown in which the estimation unit 31 is equipped with a learning unit 311, but this is not limited to the above. The vehicle 1e may be equipped with a learning device that generates a learned model separately from the vehicle 1e, the learning device may generate the learned model, and the learned model generated by the learning device may be stored in the characteristic information storage unit 40.

[0106] The information processing device 3e shown in Fig. 13 is realized by, for example, the processing circuit exemplified in Fig. 3. The identification unit 39 shown in Fig. 13 is realized by executing a program stored in the memory 102 shown in Fig. 3 by the processor 101 shown in Fig. 3. The memory 102 shown in Fig. 3 is also used to realize the identification unit 39 shown in Fig. 13. The characteristic information storage unit 40 shown in Fig. 13 is a part of the memory 102 shown in Fig. 3.

[0107] The operation of this embodiment is applicable when estimating a sleep-related index in any one of the first to fourth embodiments, or in a combination of two or more of the first to fourth embodiments.

[0108] As described above, in this embodiment, the user or the attributes of the user are identified, and based on the identification result, sleep-related indices such as whether the user is asleep or not, sleep depth, wake-up time, etc. are estimated. This provides the same effects as in the first embodiment, and can improve the estimation accuracy of the sleep-related indices.

[0109] The configurations shown in the above embodiments are merely examples, and may be combined with other known technologies, or the embodiments may be combined with each other. Also, parts of the configurations may be omitted or modified without departing from the spirit of the invention. [Explanation of symbols]

[0110] 1, 1a, 1b, 1c, 1d, 1e vehicle, 2 sensor, 3, 3a, 3b, 3d, 3e information processing device, 4 driving mechanism, 5 wakefulness promotion device, 6 information providing device, 11 speaker, 12 sound reduction device, 13 light reduction device, 30 sensor information acquisition unit, 31 estimation unit, 32 control information generation unit, 33 driving control unit, 34 information storage unit, 35, 36 communication unit, 37 input reception unit, 38 sleep information acquisition unit, 39 identification unit, 40 characteristic information storage unit, 100 vehicle system, 311 learning unit, 312 inference unit.

Claims

1. an estimation unit that estimates a sleep depth of the user while sleeping by using sensor information acquired by a sensor that detects at least one of a state of the user riding in the vehicle and an environment around the user; a control information generating unit that generates control information used for controlling autonomous driving of the vehicle by using the sleep depth estimated by the estimation unit; Equipped with The estimation unit estimates the sleep depth for a certain time ahead, the control information generation unit determines a period during which the user is predicted to be in light sleep, using the sleep depth for the certain time ahead estimated by the estimation unit, determines a route for the vehicle so as to arrive at a parking location within the determined period, and generates the control information indicating the determined route; The control information generation unit generates the control information to park the vehicle in the parking location, and after the vehicle arrives at the parking location and is parked in the parking location, if it determines that the user is in a deep sleep using the estimation result of the sleep depth estimated by the estimation unit based on the sensor information acquired while the vehicle is parked in the parking location, generates the control information indicating that the vehicle is to depart from the parking location and resume driving.

2. The information processing device according to claim 1 , wherein the parking location includes at least one of a service area and a parking area.

3. The information processing device according to claim 1, characterized in that, when the control information generation unit determines that the user is in a light sleep using the sleep depth estimated by the estimation unit, it generates the control information for making the vehicle drive more gentle than during normal times without any restrictions.

4. The information processing device according to claim 3 , wherein the gentle driving includes driving on a route with few bumps and irregularities.

5. The information processing device according to claim 3 , wherein the gentle driving includes driving on a route with few sharp curves.

6. 4. The information processing device according to claim 3, wherein the gentle driving includes driving in which at least one of the acceleration and the speed of the vehicle is equal to or lower than a threshold value.

7. 7. The information processing device according to claim 1, wherein the control information generation unit operates a sound reduction device that reduces noise inside the vehicle when the control information generation unit determines that the user is in light sleep based on the sleep depth estimated by the estimation unit.

8. 7. The information processing device according to claim 1, wherein the control information generation unit operates a dimming device to reduce light inside the vehicle when the control information generation unit determines that the user is in a light sleep based on the sleep depth estimated by the estimation unit.

9. The estimation unit predicts a wake-up time of the user by using the estimated sleep depth, 7. The information processing device according to claim 1, wherein the control information generation unit determines a route for the vehicle so that the vehicle arrives at the destination within a certain time from the wake-up time predicted by the estimation unit, and generates the control information indicating the determined route.

10. An identification unit for identifying the user; A characteristic information acquisition unit that acquires characteristic information indicating sleep-related characteristics for each user; Equipped with The information processing apparatus according to claim 9 , wherein the estimation unit predicts the wake-up time by using a result of the identification by the identification unit and the characteristic information.

11. The information processing apparatus according to claim 10 , wherein the characteristic information includes a performance value of at least one of a wake-up time and a sleep time of the user.

12. An identification unit for identifying the user; A characteristic information acquisition unit that acquires characteristic information indicating sleep-related characteristics for each user; Equipped with The information processing apparatus according to claim 1 , wherein the estimation unit estimates the sleep depth by using a result of the classification by the classification unit and the characteristic information.

13. An identification unit for identifying an attribute of the user; a characteristic information acquiring unit that acquires characteristic information indicating a sleep-related characteristic for each of the attributes; Equipped with The information processing apparatus according to claim 1 , wherein the estimation unit estimates the sleep depth by using the attribute identified by the identification unit and the characteristic information.

14. The information processing device according to claim 12, characterized in that the characteristic information is a trained model generated by supervised learning using a plurality of data sets each consisting of a feature calculated using the sensor information and a sleep depth which is corresponding ground truth data.

15. an estimation unit that estimates a sleep depth of the user using sensor information acquired by a sensor that detects at least one of a state of the user riding in the vehicle and an environment around the user; a control information generating unit that generates control information used for controlling autonomous driving of the vehicle by using the sleep depth estimated by the estimation unit; an input receiving unit that receives an input of a selection result of whether the user will get off or not at the parking location; Equipped with The estimation unit estimates the sleep depth for a certain time ahead, the control information generation unit determines a period during which the user is predicted to be in light sleep, using the sleep depth for the certain time ahead estimated by the estimation unit, determines a route for the vehicle so as to arrive at the parking location within the determined period, and generates the control information indicating the determined route; The information processing device is characterized in that, when the selection result indicates that the user will get off at a parking location, the control information generation unit controls an awakening promotion device in the vehicle to awaken the user when the vehicle arrives at the parking location, and, when the selection result indicates that the user will not get off at the parking location, the control information generation unit does not control the awakening promotion device to awaken the user even when the vehicle arrives at the parking location.

16. A vehicle capable of autonomous driving, an estimation unit that estimates a sleep depth of the user while sleeping by using sensor information acquired by a sensor that detects at least one of a state of the user riding in the vehicle and an environment around the user; a control information generating unit that generates control information used to control traveling of the vehicle, using the sleep depth estimated by the estimation unit; a driving control unit that controls the autonomous driving of the vehicle using the control information; Equipped with The estimation unit estimates the sleep depth for a certain time ahead, the control information generation unit determines a period during which the user is predicted to be in light sleep, using the sleep depth for the certain time ahead estimated by the estimation unit, determines a route for the vehicle so as to arrive at a parking location within the determined period, and generates the control information indicating the determined route; The control information generation unit generates the control information to park the vehicle in the parking location, and after the vehicle arrives at the parking location and is parked in the parking location, if it determines that the user is in a deep sleep using the estimation result of the sleep depth estimated by the estimation unit based on the sensor information acquired while the vehicle is parked in the parking location, generates the control information indicating that the vehicle is to depart from the parking location and resume driving.

17. A vehicle control method in an information processing device that controls a vehicle capable of autonomous driving, an estimation step of estimating a sleep depth of the user during sleep up to a certain time ahead, using sensor information acquired by a sensor that detects at least one of a state of the user riding in the vehicle and an environment around the user; a generating step of determining a period during which the user is predicted to be in light sleep using the estimated sleep depth for the certain time ahead, determining a route for the vehicle so as to arrive at a parking location within the determined period, and generating control information used to control the autonomous driving of the vehicle, the control information indicating the route; Including, In the generation step, control information for parking the vehicle at the parking location is generated, and after the vehicle arrives at the parking location and is parked at the parking location, if it is determined that the user is in a deep sleep using the estimation result of the sleep depth estimated in the estimation step based on the sensor information acquired while the vehicle is parked at the parking location, the control information indicating that the vehicle is to depart from the parking location and resume driving is generated.

18. A computer system that controls a vehicle capable of autonomous driving. an estimation step of estimating a sleep depth of the user during sleep up to a certain time ahead, using sensor information acquired by a sensor that detects at least one of a state of the user riding in the vehicle and an environment around the user; a generating step of determining a period during which the user is predicted to be in light sleep using the estimated sleep depth for the certain time ahead, determining a route for the vehicle so as to arrive at a parking location within the determined period, and generating control information used to control the autonomous driving of the vehicle, the control information indicating the route; Run the command, In the generation step, control information for parking the vehicle in the parking location is generated, and after the vehicle arrives at the parking location and is parked in the parking location, if it is determined that the user is in a deep sleep using the estimation result of the sleep depth estimated in the estimation step based on the sensor information acquired while the vehicle is parked in the parking location, the control information indicating that the vehicle is to depart from the parking location and resume driving is generated.

Images