Information processing device, vehicle, vehicle control method, and program

Abstract

An information processing device (3) according to the present disclosure comprises: an estimation unit (31) that estimates the sleep depth of a user riding in a vehicle (1) by using sensor information acquired by a sensor (2) that detects at least one of the state of the user and the environment surrounding the user; and a control information generation unit (32) that uses the sleep depth estimated by the estimation unit (31) to generate control information to be used to control travel of the vehicle (1) by automatic driving.

Description

Information processing device, vehicle, vehicle control method and program

[0001] The present disclosure relates to an information processing device for controlling a vehicle, a vehicle, a vehicle control method, and a program.

[0002] Users of vehicles sometimes fall asleep while riding in them. Patent Document 1 describes a technology for preventing the sleep of vehicle occupants from being disturbed in a series hybrid vehicle. The vehicle described in Patent Document 1 is capable of generating electricity using an electric motor by using engine power, and prohibits power generation around the time when the occupants start to sleep. In this way, the vehicle described in Patent Document 1 prevents changes in vibrations and sounds caused by power generation around the time when the occupants start to sleep, when their sleep is likely to be disturbed by the effects of vibrations and sounds.

[0003] Japanese Patent Application Laid-Open No. 2019-131109

[0004] The technology described in Patent Document 1 prevents changes in vibrations and sounds caused by power generation before and after the occupant starts to sleep, but events that disrupt sleep are not limited to changes in vibrations and sounds caused by power generation; the vehicle's running itself can also affect the occupant. For this reason, it is desirable to perform running control that corresponds to the user's sleep status.

[0005] The present disclosure has been made in view of the above, and aims to provide an information processing device that can perform vehicle driving control according to a user's sleep state.

[0006] In order to solve the above-mentioned problems and achieve the objectives, the information processing device of the present disclosure includes an estimation unit that estimates the depth of sleep of a user using sensor information acquired by a sensor that detects at least one of the state of a user riding in a vehicle and the environment surrounding the user, and a control information generation unit that generates control information used to control the vehicle's autonomous driving using the depth of sleep estimated by the estimation unit.

[0007] The information processing device according to the present disclosure advantageously enables vehicle driving control to be performed in accordance with the sleep-related state of the user.

[0008] FIG. 1 is a diagram showing an example of the configuration of a vehicle according to the first embodiment; FIG. 2 is a diagram showing another example of the configuration of the first embodiment; FIG. 3 is a diagram showing an example of the configuration of a processing circuit realizing the information processing device of the first embodiment; FIG. 4 is a flowchart showing an example of the processing procedure in the information processing device of the second embodiment;

[0009] An information processing device, a vehicle, a vehicle control method, and a program according to embodiments will be described in detail below with reference to the accompanying drawings.

[0010] First Embodiment Fig. 1 is a diagram showing an example of the configuration of a vehicle according to a first embodiment. Vehicle 1 of this embodiment is an autonomous vehicle, and includes a sensor 2, an information processing device 3, and a driving mechanism 4. Note that, although an example in which vehicle 1 is an autonomous vehicle will be described here, vehicle 1 may be capable of switching between autonomous driving and manual driving, and may be any vehicle capable of autonomous driving.

[0011] The sensor 2 is a detection device that detects at least one of the state of a user riding in the vehicle 1 and the user's surrounding environment, and transmits the detection result as sensor information to the information processing device 3. The sensor information is information indicating at least one of the state of a user riding in the vehicle 1 and the user's surrounding environment. The sensor 2 includes, but is not limited to, at least one of a sensor that acquires the user's vital data, a camera that captures images of the user, a sensor that detects sound, an acceleration sensor attached to the user, an acceleration sensor that detects the acceleration of the vehicle 1, and a sensor attached to the vehicle 1. The vital data includes, but is not limited to, at least one of heart rate, body temperature, blood pressure, and respiratory rate. The sensor 2 that acquires the vital data may be worn by the user, a non-contact sensor fixed to the vehicle 1, or other sensors. Note that while FIG. 1 illustrates one sensor 2, multiple sensors 2 may be used, and the number of sensors 2 is not limited to the example shown in FIG. 1.

[0012] The information processing device 3 includes a sensor information acquisition unit 30, an estimation unit 31, a control information generation unit 32, a driving control unit 33, and an information storage unit 34. The sensor information acquisition unit 30 acquires sensor information by receiving it from the sensor 2, and outputs the acquired sensor information to the estimation unit 31. The estimation unit 31 estimates the depth of sleep of the user using the sensor information received from the sensor information acquisition unit 30, i.e., the sensor information acquired by the sensor 2. For example, the estimation unit 31 estimates whether the user is awake or asleep using the sensor information, and if the user is asleep, estimates the depth of sleep of the user and notifies the estimated depth of sleep to the control information generation unit 32.

[0013] Sleep depth is an index indicating the depth of sleep, and may be indicated, for example, by two stages: REM sleep, which is light sleep, and non-REM sleep, which is deeper than REM sleep, or non-REM sleep may be further divided into three or four stages, for a total of four or five stages. Generally, REM sleep and non-REM sleep alternate. The repeated cycle of REM sleep and non-REM sleep is also called a sleep cycle, and the sleep cycle is, for example, about 90 to 120 minutes. REM sleep is a sleep stage in which light sleep occurs and rapid eye movement occurs. In REM sleep, breathing, pulse rate, blood pressure, etc. are high compared to non-REM sleep. Non-REM sleep is deeper than REM sleep, and as described above, non-REM sleep is also classified into three or four stages depending on the depth of sleep. For example, if non-REM sleep is classified into four stages, from stage 1, which is the lightest, to stage 4, which is the deepest, then, after falling asleep, a person generally goes through stages 1, 2, 3, and 4 of non-REM sleep, which become deeper in that order. After stage 4 of non-REM sleep, REM sleep appears. After that, REM sleep and non-REM sleep are repeated, but in the latter half of sleep, the amount of lighter stages, i.e., stages 1 and 2 of non-REM sleep, increases, and REM sleep also increases.

[0014] The estimation unit 31 can estimate whether the sleep depth is one of four stages (REM sleep or non-REM sleep) or one of three stages based on sleep parameters, such as vital data or an indicator of body movement. The indicator of body movement may be, but is not limited to, the number of times the user turns over in sleep or the number of times a specific location moves by a certain amount or more. The indicator of body movement may be, but is not limited to, detected by a camera that captures the user, an acceleration sensor attached to the user, or a sensor (such as an acceleration sensor or a pressure sensor) attached to the vehicle 1. For example, the correspondence between the sleep depth and the numerical range of the sleep parameter may be stored in a table format as reference information, and the estimation unit 31 may estimate the sleep depth using the sensor information and the reference information. Alternatively, the relationship between the sleep parameter and the sleep depth may be learned by machine learning, and the estimation unit 31 may estimate the sleep depth using the learning result and the acquired sleep parameter. Furthermore, the estimation unit 31 may further estimate the depth of sleep based on the characteristics of the changes between the four or three stages of REM sleep and non-REM sleep in the above-mentioned typical human sleep (for example, a standard pattern of repetition between the four or three stages of REM sleep and non-REM sleep from falling asleep to waking up).

[0015] Generally, the third and fourth stages of non-REM sleep are deep sleep, and during deep sleep, external stimuli such as sound, vibration, and light are less likely to disturb sleep, and external stimuli have less impact on sleep rhythm. On the other hand, the first and second stages of non-REM sleep are light sleep, and compared to deep sleep, they are more susceptible to external stimuli, and events such as sleep disturbance and sleep rhythm disruption due to external stimuli are more likely to occur. REM sleep can be considered a light sleep that is easy to wake up from, but is less susceptible to external stimuli. Therefore, whether REM sleep is light sleep or deep sleep can be appropriately set depending on the purpose of sleep depth. Note that the definitions of deep sleep and light sleep are not limited to this example, and can be appropriately set depending on the purpose of sleep depth.

[0016] Although sensor information indicating the user's surrounding environment does not directly indicate the user's state, it may be used to determine whether the surrounding environment is suitable for sleep. For example, the estimation unit 31 may use the sensor information indicating the user's surrounding environment to determine whether the environment is unsuitable for sleep and use the determination result to estimate the sleep depth. For example, as described below, assume that sleep-related driving modes such as a mode to promote sleep and a mode for when the user is awake are set, and acceleration is limited in the sleep-promoting mode. In this case, the estimation unit 31 can estimate that the sleep-promoting mode is selected using sensor information acquired by the sensor 2, which is an acceleration sensor that detects the acceleration of the vehicle 1, thereby indirectly grasping the user's sleep state.

[0017] The information storage unit 34 stores map information and route information. The map information may be two-dimensional map information, three-dimensional map information, or a dynamic map that includes dynamic information. When the dynamic information of the dynamic map is received, for example, by a communication unit (not shown), the dynamic information stored in the information storage unit 34 is updated with the received dynamic information. The route information is information that indicates the route of the vehicle 1, and may include the required time (or estimated arrival time) to the destination, the estimated arrival time at intermediate points, etc.

[0018] The control information generation unit 32 uses the sleep depth estimated by the estimation unit 31 to generate control information used to control the autonomous driving of the vehicle 1, and outputs the generated control information to the driving control unit 33. The driving control of the vehicle 1 is the control of the driving itself of the vehicle 1, such as setting a travel route, controlling the speed of the vehicle 1, controlling the acceleration of the vehicle 1, controlling handling, and controlling the stopping of the vehicle 1. The control information generation unit 32 may, for example, use the sleep depth estimated by the estimation unit 31 to determine whether the user is in deep sleep, and if the user is not in deep sleep, generate control information to reduce external stimuli.

[0019] Methods for reducing external stimuli include, but are not limited to, at least one of selecting a route (driving route) with fewer external stimuli, suppressing sudden acceleration and deceleration, suppressing speed, and gentle handling. Furthermore, for example, the estimation unit 31 may estimate the sleep depth for a certain period using the estimated sleep depth, i.e., predict the sleep depth for the certain period, and notify the control information generation unit 32 of the estimated sleep depth for the certain period. In this case, the control information generation unit 32 may use the sleep depth for the certain period to set a route that allows the user to wake up as naturally as possible at a point where the user is supposed to wake up. The certain period is a future period, for example, a period from the present time. The point where the user is supposed to wake up may be the user's destination or a place where the user plans to stop along the way.

[0020] The control information generation unit 32 may also switch the driving mode (hereinafter also referred to as the driving mode) using the sleep depth for a certain period of time. For example, when the user is awake and not asleep, the driving mode may be set to normal driving without any particular restrictions (normal driving mode); when the user is in light sleep, the driving mode may be set to a restricted driving mode that imposes restrictions on gentle driving; and when the user is in deep sleep, the driving mode may be set to the normal driving mode. Gentle driving is driving that reduces external stimuli for the user. The definition and switching method of the driving mode are not limited to this example. Note that when the control information generation unit 32 generates control information related to a route, the control information generation unit 32 may also generate the control information using map information stored in the information storage unit 34.

[0021] Furthermore, the control information generation unit 32 may generate control information for controlling devices (not shown) in the vehicle 1 in addition to control information for controlling driving, thereby controlling the devices. For example, the control information generation unit 32 may control devices in the vehicle 1, such as seats, lighting adjustment equipment, devices that generate sounds such as music or environmental sounds, and massage devices installed in the seats, according to the depth of sleep. For example, if the user is in a light sleep state, the control unit 32 may control the seat to gently recline or the volume of a sound-generating device to lower so as to promote sleep. Furthermore, if the required time to the destination is long, control may be performed to promote sleep, such as operating a massage device or playing relaxing music, to promote sleep before the user falls asleep. Furthermore, to promote sleep, the control information generation unit 32 may set the above-mentioned driving mode to a limited driving mode.

[0022] The user may be able to set whether or not to perform control to promote sleep onset. Furthermore, the control information generator 32 may, for example, set a wake-up time desired by the user, calculate backward from the set wake-up time, determine a sleep onset timing so that the user will be in REM sleep or light non-REM sleep at the wake-up time, and start sleep promotion before the determined timing, thereby making it easier for the user to fall asleep at or close to the determined timing. Alternatively, the control information generator 32 may set the wake-up time to the estimated arrival time at the destination, determine a sleep onset timing so that the user will be more likely to wake up near the destination, and similarly perform control to promote sleep onset.

[0023] The driving control unit 33 controls the autonomous driving of the vehicle 1. The driving control unit 33 controls the driving of the vehicle 1 by controlling the driving mechanism 4 based on information acquired by sensors (not shown), such as a camera for detecting obstacles, a light detection and ranging (LiDAR), and a millimeter-wave sensor, map information stored in the information storage unit 34, and a self-position calculated by a self-position estimation device (not shown). For example, when a destination is set by a user or another operator, the driving control unit 33 uses the destination and map information to set a route to the set destination and controls the driving mechanism 4 to drive along the set route. The driving control unit 33 stores route information indicating the set route in the information storage unit 34. The driving control unit 33 may also calculate the required time (or estimated arrival time) to the destination and the estimated arrival time at intermediate points, and include these in the route information. The driving control unit 33 may set the route using dynamic information related to roads and traffic, such as congestion information and obstacle information. Note that the driving control for automatic driving may be performed by any method, and a general method can be used, so detailed explanation will be omitted.

[0024] Furthermore, upon receiving control information from the control information generator 32, the traveling control unit 33 controls the traveling mechanism 4 based on the received control information. Note that, although an example in which the traveling control unit 33 sets a route will be described here, the control information generator 32 may set a route and notify the traveling control unit 33 of the set route as control information, and the traveling control unit 33 may control the traveling mechanism 4 according to the route based on the control information.

[0025] The driving mechanism 4 is a mechanism for driving the vehicle 1, and includes a plurality of mechanisms for driving the vehicle 1, such as an accelerator operating device such as an accelerator pedal, a steering device, and a brake.

[0026] In this embodiment, the information processing device 3 estimates the depth of sleep of the user and generates control information for controlling the driving of the vehicle 1 based on the estimated depth of sleep. This allows the vehicle 1 to control the driving of the vehicle 1 in accordance with the user's sleep state. Therefore, the vehicle 1 can make the user's environment, which is caused by the acceleration, deceleration, handling, and driving route of the vehicle 1, suitable for the user's depth of sleep. Furthermore, the vehicle 1 can achieve a natural awakening of the user by controlling the time the user should wake up so that the user is in REM sleep or light non-REM sleep.

[0027] In the example described above, the information processing device 3 is provided in the vehicle 1, but an information processing device may be provided separately from the vehicle. FIG. 2 is a diagram showing another configuration example of this embodiment. In the example shown in FIG. 2, the vehicle system 100 includes an information processing device 3a and a vehicle 1a. Components having the same functions as those in the example shown in FIG. 1 are assigned the same reference numerals as in FIG. 1, and redundant description will be omitted. The information processing device 3a is provided separately from the vehicle 1a, and a communication unit 35 capable of communicating with the vehicle 1a is added instead of the sensor information acquisition unit 30 of the information processing device 3 shown in FIG. 1. The communication unit 35 also functions as a sensor information acquisition unit that acquires sensor information by receiving sensor information. In addition, the driving control unit 33 in the information processing device 3 shown in FIG. 1 is provided in the vehicle 1a.

[0028] 1 , the vehicle 1a has the information processing device 3 removed and a communication unit 36 ​​capable of communicating with the information processing device 3a added. The communication line between the vehicle 1a and the information processing device 3a includes, for example, a wireless communication line, but may also be a combination of a wireless communication line and a wired communication line. Furthermore, communication between the vehicle 1a and the information processing device 3a may be performed via another device.

[0029] The communication unit 36 ​​of the vehicle 1a receives sensor information from the sensor 2 and transmits the received sensor information to the information processing device 3a. The communication unit 35 of the information processing device 3a receives sensor information from the vehicle 1a and outputs the received sensor information to the estimation unit 31. The operations of the estimation unit 31 and the control information generation unit 32 are similar to those in the example shown in FIG. 1 , but the control information generation unit 32 outputs the generated control information to the communication unit 35. The communication unit 35 transmits the control information received from the control information generation unit 32 to the vehicle 1a. When the communication unit 36 ​​of the vehicle 1a receives control information from the information processing device 3a, it outputs the received control information to the driving control unit 33. The driving control unit 33, the information storage unit 34, and the driving mechanism 4 are similar to those in the example shown in FIG. 1 .

[0030] In addition, when the control information generation unit 32 generates control information regarding a route, the information processing device 3a may further include an information storage unit 34, and may generate the control information using map information stored in the information storage unit 34.

[0031] Next, the hardware configuration of the information processing device 3 of this embodiment will be described. In the information processing device 3 of this embodiment, a computer system executes a program (computer program) describing the processing in the information processing device 3, causing the computer system to function as the information processing device 3. The computer system includes, for example, a processing circuit. FIG. 3 is a diagram showing an example configuration of the processing circuit that realizes the information processing device 3 of this embodiment. As shown in FIG. 3, the processing circuit includes a processor 101, a memory 102, and a communication unit 103. The processing circuit may be a single circuit or multiple circuits, which are connected via a system bus. The communication unit 103 may be provided separately from the processing circuit. The computer system that realizes the information processing device 3 may further include at least one of an input unit, such as a button or keyboard, that accepts input, and a display unit, such as a display or monitor.

[0032] 3, the processor 101 is a control unit such as a CPU (Central Processing Unit) that executes a program describing the processing in the information processing device 3 of this embodiment. The memory 102 includes various types of memory such as a RAM (Random Access Memory) and a ROM (Read Only Memory) and a storage device such as a hard disk, and stores the program to be executed by the processor 101, necessary data obtained during the processing, etc. The memory 102 is also used as a temporary storage area for the program. The communication unit 103 is a receiver and transmitter that perform communication processing.

[0033] Here, an example of the operation of the computer system until the program of this embodiment is ready to be executed will be described. In the computer system having the above configuration, for example, the program is installed into memory 102 from a CD-ROM or DVD-ROM inserted in a CD (Compact Disc)-ROM drive or DVD (Digital Versatile Disc)-ROM drive (not shown). Then, when the program is executed, the program is read from memory 102 and stored in the main storage area of ​​memory 102. In this state, processor 101 executes the processing as information processing device 3 of this embodiment in accordance with the program stored in memory 102.

[0034] In the above explanation, a program describing the processing in the information processing device 3 is provided using a CD-ROM or DVD-ROM as a recording medium, but this is not limited to this. Depending on the configuration of the computer system, the capacity of the program to be provided, etc., it is also possible to use a program provided via a transmission medium such as the Internet via the communication unit 103.

[0035] The program of this embodiment, for example, causes a computer system that controls vehicle 1 to execute the steps of estimating the user's sleep depth using sensor information acquired by sensor 2, and generating control information that is used to control the autonomous driving of vehicle 1 using the estimated sleep depth.

[0036] The estimation unit 31, control information generation unit 32, and driving control unit 33 shown in Fig. 1 are realized by the processor 101 shown in Fig. 3 executing programs stored in the memory 102 shown in Fig. 3. The memory 102 shown in Fig. 3 is also used to realize the estimation unit 31, control information generation unit 32, and driving control unit 33 shown in Fig. 1. The sensor information acquisition unit 30 shown in Fig. 1 is realized by the communication unit 103 shown in Fig. 3. The information storage unit 34 shown in Fig. 1 is part of the memory 102 shown in Fig. 3.

[0037] Similarly, the information processing device 3a shown in Fig. 2 is realized by a computer system including the processing circuit shown in Fig. 3. Note that the computer system realizing the information processing device 3a may further include at least one of an input unit and a display unit in addition to the processing circuit. The communication unit 35 shown in Fig. 2 is realized by the communication unit 103 shown in Fig. 3. The information processing device 3a may also be realized by multiple computer systems. For example, the information processing device 3a shown in Fig. 2 may be realized by a cloud system.

[0038] Embodiment 2 In embodiment 2, an example of driving control of a vehicle 1 by the information processing device 3 described in embodiment 1 will be described. The configuration of the vehicle 1 in this embodiment is the same as that in embodiment 1. Note that, although the description here takes as an example a configuration in which the information processing device 3 is provided inside the vehicle 1 as shown in FIG. 1 of embodiment 1, the operation of this embodiment can also be similarly applied to the configuration example shown in FIG. 2 of embodiment 1. Components having the same functions as those in embodiment 1 are assigned the same reference numerals as those in embodiment 1, and redundant description will be omitted.

[0039] In this embodiment, the information processing device 3 sets a route to the destination so that the vehicle 1 arrives at a parking spot such as a service area (SA) and parks there, and then resumes traveling, when the user of the vehicle 1 is in a light sleep state. This allows the information processing device 3 to prevent the user from being stimulated by the traveling of the vehicle 1 when the user is in a light sleep state, and promotes deep sleep for the user.

[0040] 4 is a flowchart showing an example of a processing procedure in the information processing device 3 according to the present embodiment. As shown in FIG. 4, the information processing device 3 determines whether the user is asleep (step S1). In the present embodiment, the sensor information acquisition unit 30 acquires sensor information from the sensor 2, for example, periodically.

[0041] In step S1, specifically, the estimation unit 31 determines whether the user of the vehicle 1 is sleeping using the sensor information received from the sensor information acquisition unit 30. The determination of whether the user is sleeping is performed, for example, using a camera as the sensor 2 and using image data obtained by the camera capturing an image of the user. The estimation unit 31 may, for example, detect the user's movement based on the image data, which is sensor information, and determine whether the user is sleeping based on the user's movement. Alternatively, the estimation unit 31 may determine whether the user's eyelids are closed based on the image data and determine whether the user is sleeping based on the determination result of whether the eyelids are closed. Alternatively, the sensor 2 may be an acceleration sensor worn by the user, and the estimation unit 31 may determine whether the user is sleeping based on sensor information, which is the detection result of the acceleration sensor. Alternatively, the sensor 2 may be a sensor that acquires vital data, and the estimation unit 31 may determine whether the user is sleeping based on sensor information, which is vital data. Alternatively, the estimation unit 31 may determine whether the user is sleeping by combining the image data and the vital data. The method of determining whether the user is asleep is not limited to these examples.

[0042] If the user is not asleep (step S1: No), step S1 is repeated. If the user is asleep (step S1: Yes), the information processing device 3 estimates the sleep depth to the destination (step S2). Specifically, the estimation unit 31 extracts the route and required time to the destination from the route information stored in the information storage unit 34, and uses the extracted information to estimate the sleep depth at each time from the present to the time of arrival at the destination, and outputs the estimated sleep depth to the control information generation unit 32. Note that, as described in the first embodiment, the route and required time to the destination are calculated, for example, by the driving control unit 33 and stored in the information storage unit 34 as route information. Also, as described in the first embodiment, the control information generation unit 32 may set a route using the destination and map information and calculate the required time. In this case, the control information generation unit 32 stores the route information in the information storage unit 34. Note that, although an example in which the estimation unit 31 estimates the depth of sleep up to the destination will be described here, the estimation unit 31 is not limited to this, and may estimate the depth of sleep up to a certain time ahead. As described above, the certain time may be the time required to reach the destination, a time shorter than the time required to reach the destination, such as half the time required to reach the destination, or a predetermined time.

[0043] Next, the information processing device 3 sets a route to arrive at the parking location during a light sleep period (step S3). Specifically, the control information generation unit 32 determines a period during which the user is predicted to be in light sleep, using the depth of sleep to the destination estimated by the estimation unit 31, determines a route for the vehicle 1 so that the vehicle 1 arrives at the parking location within the determined period, and generates control information indicating the determined route. This control information may be output to the driving control unit 33 or may be stored as route information in the information storage unit 34.

[0044] For example, the control information generation unit 32 first calculates the position of the vehicle 1 during the light sleep period using the route to the destination and the sleep depth to the destination received from the estimation unit 31. If a parking space is available near the calculated position, the control information generation unit 32 determines a route to arrive at the parking space during that period and park the vehicle there. The parking space may include, for example, at least one of an SA and a PA (parking area), but may also include a rest area such as a roadside station or a parking lot where parking is permitted. The parking lot may be a parking lot for a store, park, or the like. If a parking space is available on the initially set route, the control information generation unit 32 may determine a driving speed to arrive at the parking space during the light sleep period, and may output the determined driving speed to the driving control unit 33 as control information. If a parking space is not available on the initially set route, the control information generation unit 32 selects a parking space nearby the route that will allow the vehicle to arrive during the light sleep period, and sets a route that passes through the selected parking space. That is, the control information generation unit 32 updates the route information stored in the information storage unit 34 to route information (control information) indicating a route that passes through the parking location. Alternatively, the control information generation unit 32 may output the route information indicating the route that passes through the parking location to the driving control unit 33 as control information. Furthermore, the control information generation unit 32 generates control information for parking the vehicle 1 at the parking location and outputs the generated control information to the driving control unit 33. Note that an instruction to park the vehicle 1 at the parking location may be included in the route information. As a result, the driving control unit 33 controls the driving of the vehicle 1 based on the updated route information, and when the vehicle arrives at the parking location, controls the driving mechanism 4 to park the vehicle 1 at the parking location.

[0045] If it is estimated that light sleep periods will occur a predetermined number of times or more before arriving at the destination, it is possible to determine parking locations corresponding to two or more of the light sleep periods, assuming that the vehicle will park in a parking location for a certain period of time before resuming driving, and to set a route that passes through the two or more determined parking locations. For example, if the required time to the destination is seven hours or more, four to five sleep cycles will occur, and the latter sleep cycles are preparations for awakening, so the control information generator 32 may set a route that passes through two parking locations so as to promote deep sleep in the first two sleep cycles.

[0046] Next, the information processing device 3 determines whether the vehicle 1 has arrived at the parking location (step S4). In detail, the control information generating unit 32 may determine that the vehicle 1 has arrived at the parking location by receiving a notification from the driving control unit 33 that the vehicle 1 has been parked in the parking location, or may obtain the current position of the vehicle 1 from the driving control unit 33 and determine that the vehicle 1 has arrived at the parking location if the current position is within the parking location.

[0047] If the user has not arrived at the parking space (step S4: No), step S4 is repeated. If the user has arrived at the parking space (step S4: Yes), the information processing device 3 estimates the sleep depth (step S5). Specifically, the control information generation unit 32 instructs the estimation unit 31 to estimate the sleep depth, and the estimation unit 31 estimates the user's current sleep depth using the sensor information and outputs the estimation result to the control information generation unit 32.

[0048] The information processing device 3 determines whether the user is in deep sleep (step S6). In detail, the control information generating unit 32 determines whether the user is in deep sleep based on the estimation result received from the estimation unit 31.

[0049] If the user is not in deep sleep (step S6: No), the processes from step S5 are repeated. If the user is in deep sleep (step S6: Yes), the information processing device 3 resumes driving (step S7). Specifically, the control information generation unit 32 generates control information instructing the user to resume driving and outputs the generated control information to the driving control unit 33. The driving control unit 33 controls the driving mechanism 4 to resume driving of the vehicle 1 based on the control information received from the control information generation unit 32. In this way, after the vehicle 1 arrives at the parking space, if the control information generation unit 32 determines that the user is in deep sleep using the estimation result of the sleep depth estimated by the estimation unit 31 based on the sensor information, the control information generation unit 32 generates control information indicating that the vehicle 1 is to depart from the parking space and resume driving.

[0050] After resuming driving, the information processing device 3 determines whether parking is unnecessary (step S8). Specifically, the control information generation unit 32 determines that parking is unnecessary if the vehicle has already parked at all parking locations along the route indicated by the route information. Note that even if there are parking locations that have not yet been parked among all parking locations along the route indicated by the route information, the control information generation unit 32 may determine that parking is unnecessary in step S8 even if there are parking locations that have not yet been parked, if the drive is expected to take longer than originally planned and arrival at the destination is expected to be delayed by more than a specified time.

[0051] If parking is not necessary (step S8: No), the process from step S4 is repeated. If parking is not necessary (step S8: Yes), the information processing device 3 drives the vehicle 1 to the destination (step S9). Specifically, in step S9, if the vehicle has already parked at all the parking spaces to be passed through, the driving control unit 33 controls the driving mechanism 4 to drive the vehicle 1 to the destination based on the route information. If it is determined in step S8 that parking is not necessary even though there are parking spaces that have not yet been parked, in step S9, the control information generation unit 32 sets a route that does not pass through the remaining parking spaces that were planned to be passed through (parking spaces that have not yet been reached). That is, the control information generation unit 32 updates the route information stored in the information storage unit 34 to route information that indicates a route that does not pass through the remaining parking spaces. As a result, the driving control unit 33 controls the driving of the vehicle 1 based on the updated route information, thereby driving the vehicle 1 to the destination.

[0052] 4, the information processing device 3 resumes driving when the user falls into a deep sleep after the vehicle 1 is parked in a parking space, but the invention is not limited to this and the information processing device 3 may resume driving after the vehicle 1 has been parked in a parking space for a predetermined time. Furthermore, the information processing device 3 may resume driving when at least one of the conditions that the user falls into a deep sleep and that the vehicle has been parked in a parking space for a predetermined time is satisfied.

[0053] FIG. 5 is a conceptual diagram illustrating an example of route setting in this embodiment. In the example shown in FIG. 5, a route 201 from a departure point to a destination is set. Assume that the user falls asleep at point 202 on the route 201 after the vehicle 1 departs from the departure point. In the example shown in FIG. 5, the information processing device 3 predicts the user's sleep depth, determines that the user is in light sleep during the time (period) while traveling through section 203, and sets the route 201 to pass through SA301 and SA302. Section 204 corresponds to the time (period) during which the user is in deep sleep. In this way, the vehicle 1 is parked at parking locations such as SA301 and SA302 during the user's light sleep, thereby preventing the user from being stimulated by the vehicle 1 traveling and promoting deep sleep for the user. Note that FIG. 5 is an example, and the number of set routes and parking locations is not limited to the example shown in FIG. 5. Furthermore, the parking locations are not limited to SAs, but may be PAs as described above, or parking lots for stores, parks, etc.

[0054] In the above example, parking in a parking space promotes deep sleep for the user, but it is also conceivable that the user may want to take a toilet break, go shopping, or get off at the parking space. For this reason, the user may be able to set whether deep sleep is promoted or the user is woken up at the parking space.

[0055] Fig. 6 is a diagram showing an example of the configuration of a vehicle according to this embodiment when user settings are accepted. Vehicle 1b shown in Fig. 6 is similar to vehicle 1 shown in Fig. 1 except that it has an awakening promotion device 5 added thereto and includes an information processing device 3b instead of information processing device 3. Components having the same functions as those in the example shown in Fig. 1 are assigned the same reference numerals as in Fig. 1, and redundant explanations will be omitted.

[0056] As shown in FIG. 6 , the information processing device 3 b includes an input receiving unit 37 added to the information processing device 3 shown in FIG. 1 . The input receiving unit 37 receives a selection input from the user indicating whether the user will get off at the parking location or not. The input receiving unit 37 outputs the received selection to the control information generating unit 32. If the selection indicates that the user will get off at the parking location, the control information generating unit 32 sets the mode of operation at the parking location to awakening promotion. If the mode of operation at the parking location is set to awakening promotion, the control information generating unit 32 generates control information instructing the user to awaken upon arrival at the parking location and outputs the generated control information to the awakening promotion device 5. Upon receiving the control information from the control information generating unit 32, the awakening promotion device 5 performs an operation to awaken the user. The operation to awaken the user may be, for example, an operation to generate a sound such as a voice, an alarm, or music notifying the user that the user has arrived at the parking location, or an operation to vibrate the seat, or may be other operations. If the selection result indicates that the user will not get off at the parking location, the control information generation unit 32 sets the mode of operation at the parking location to sleep promotion, and does not perform control to wake the user up even when the user arrives at the parking location, as in the example shown in Figure 4.

[0057] 7 is a flowchart showing an example of a processing procedure in the information processing device 3b according to this embodiment when accepting a user setting. Steps S1 to S4 are the same as the example shown in FIG. 4. If the answer to step S4 is Yes, the information processing device 3b determines whether the setting is for waking up (step S11). Specifically, the control information generator 32 determines whether the mode of operation in the parking space is for waking up. If the setting is not for waking up (step S11 No), steps S5 to S9 are performed as in FIG. 4.

[0058] If the setting is for awakening promotion (Yes in step S11), the information processing device 3b promotes awakening of the user (step S12). Specifically, the control information generation unit 32 generates control information instructing to awaken the user and outputs the generated control information to the awakening promotion device 5. As a result, upon receiving the control information from the control information generation unit 32, the awakening promotion device 5 performs an operation to promote awakening of the user.

[0059] Next, the information processing device 3b determines whether a departure instruction, i.e., an instruction to depart vehicle 1b, has been received (step S13). If a departure instruction has been received (step S13: Yes), the process proceeds to step S7. If a departure instruction has not been received (step S13: No), the information processing device 3b repeats step S13. Note that, for example, the instruction to depart vehicle 1b is input by the user. When the input receiving unit 37 receives an input from the user instructing vehicle 1b to depart, it outputs information indicating that the departure of vehicle 1b has been instructed to the control information generating unit 32. In step S13, the control information generating unit 32 determines whether a departure instruction has been received based on whether information indicating that the departure of vehicle 1b has been instructed from the input receiving unit 37 has been received.

[0060] In the example shown in FIG. 7 , when the operation mode is set to sleep promotion, the same processing as in FIG. 4 is performed. When the operation mode is set to awakening promotion, the user is awakened upon arrival at the parking spot. Even when waking up the user at the parking spot is promoted, the route is set so that the user arrives at the parking spot during a light sleep period, which makes it easier for the user to awaken naturally. In the example shown in FIG. 7 , the user can set the operation mode, so the information processing device 3b can operate according to the user's request. Note that an input receiving unit 37 and an awakening promotion device 5 may be added to the vehicle 1a shown in FIG. 2 of the first embodiment, and information received by the input receiving unit 37 may be transmitted to the information processing device 3a via the communication unit 36, so that the information processing device 3a can perform processing similar to that shown in FIG. 7 .

[0061] 6 and 7, vehicle 1b is equipped with an awakening promotion device 5, but vehicle 1b does not have to be equipped with an awakening promotion device 5. Since there is a possibility that the user will sense the arrival of the vehicle at the parking space when the user is in a light sleep state and wake up naturally, information processing device 3b may wait for a certain period of time after arriving at the parking space for the user to get out of the vehicle. If the user does not wake up after the certain period of time has passed, the information processing device 3b may perform an operation that would be performed if the operation mode was set to sleep promotion.

[0062] The information processing device 3b shown in FIG. 6 is realized by, for example, the processing circuit exemplified in FIG. 3 and an input unit (not shown), and the input receiving unit 37 is realized by the input unit.

[0063] As described above, in this embodiment, the route of the vehicle 1, 1b is set so that the vehicle 1, 1b arrives at the parking location during a light sleep time, and the vehicle 1, 1b stops at the parking location when the vehicle 1, 1b arrives. This prevents the user from being disturbed by the stimulation caused by the vehicle 1, 1b's movement during a light sleep time, and promotes deep sleep for the user. In other words, driving control can be performed according to the user's sleep state.

[0064] Embodiment 3. In Embodiment 3, as in Embodiment 2, an example of driving control of a vehicle 1 by the information processing device 3 described in Embodiment 1 will be described. The configuration of the vehicle 1 in this embodiment is the same as that in Embodiment 1. Note that, although the description here takes as an example a configuration in which the information processing device 3 is provided inside the vehicle 1 as shown in FIG. 1 of Embodiment 1, the operation of this embodiment can also be similarly applied to the configuration example shown in FIG. 2 of Embodiment 1. Components having the same functions as those in Embodiment 1 are assigned the same reference numerals as those in Embodiment 1, and redundant description will be omitted.

[0065] 8 is a flowchart showing an example of a processing procedure in the information processing device 3 of this embodiment. As shown in Fig. 8, the information processing device 3 determines whether the user is asleep, similarly to step S1 in embodiment 2. Note that in this embodiment as well, the sensor information acquisition unit 30 acquires sensor information from the sensor 2, for example, periodically.

[0066] If the user is not asleep (step S1 No), the information processing device 3 determines that the vehicle 1 is to be driven in normal driving mode (step S21), and step S1 is repeated. For example, as described in the first embodiment, the user can set the driving mode of the vehicle 1 to two types of modes, the normal driving mode and the restricted driving mode, and the control information generator 32 sets the driving mode to the normal driving mode. Note that if the driving mode is already set to the normal driving mode, there is no need to change the driving mode, and the control information generator 32 keeps the driving mode as the normal driving mode.

[0067] If the user is asleep (Yes in step S1), the information processing device 3 estimates the depth of sleep (step S22). Specifically, the estimation unit 31 estimates the user's current depth of sleep using sensor information and outputs the estimation result to the control information generation unit 32. Alternatively, the estimation unit 31 may predict the depth of sleep for a certain time period ahead using the estimation result of the user's current depth of sleep and output the prediction result to the control information generation unit 32.

[0068] Next, the information processing device 3 determines whether the user is in light sleep (step S23). Specifically, the control information generation unit 32 determines whether the user is in light sleep based on the estimation result received from the estimation unit 31. If the user is in light sleep (step S23: Yes), the information processing device 3 determines to restrict the driving of the vehicle 1 (step S24) and repeats the process from step S22. Specifically, in step S24, the control information generation unit 32 generates control information for making the driving of the vehicle 1 gentler than normal driving without restrictions. For example, the control information generation unit 32 sets the driving mode to the restricted driving mode. Note that if the driving mode is already set to the restricted driving mode, the driving mode does not need to be changed. The control information generation unit 32 generates control information instructing the driving mode to remain in the restricted driving mode and outputs the generated control information to the driving control unit 33. In the restricted driving mode, as described in the first embodiment, driving control is performed to ensure gentle driving.

[0069] Specifically, in the limited driving mode, the vehicle speed and / or acceleration may be limited to a threshold value or less, or the driving route may be set to a route that allows for gentle driving. Examples of routes that allow for gentle driving include, but are not limited to, routes with few sharp curves, routes with few road irregularities, and routes with gentle gradients. For example, map information may include values ​​indicating the curvature of a curve, the unevenness of a road surface, and the gradient of a road. The control information generator 32 may determine a curve as being sharp when the curvature of a curve is equal to or greater than a threshold value, determine a route with few road irregularities when the value indicating the unevenness of a road surface is equal to or less than a threshold value, and determine a route with a gentle gradient when the value indicating the gradient of a road is equal to or less than a threshold value. As described above, the control information generator 32 may receive a predicted value of sleep depth for a certain time period ahead from the estimation unit 31. In this case, the driving mode for a period in which the predicted result indicates light sleep is set to the limited driving mode.

[0070] If the user is not in light sleep (step S23: No), the information processing device 3 determines that the vehicle 1 is to be driven in normal driving mode (step S25), and repeats the process from step S22. Specifically, in step S25, for example, the control information generator 32 sets the driving mode to the normal driving mode. If the driving mode is already set to the normal driving mode, the driving mode does not need to be changed, and the control information generator 32 keeps the driving mode as the normal driving mode.

[0071] By performing the process illustrated in FIG. 8 , the information processing device 3 can control the vehicle 1 to travel smoothly during the user's light sleep hours. That is, the information processing device 3 can control the vehicle 1 to travel smoothly during the user's light sleep hours. In the above example, the vehicle 1 travels smoothly during the user's light sleep hours to avoid disturbing the user's sleep. However, the information processing device 3 may also suppress external stimuli to the user that are not related to the vehicle 1's travel during the light sleep hours. For example, the control information generator 32 may suppress external stimuli by adjusting at least one of the ambient brightness and ambient sound as the user's surrounding environment. For example, in step S24 described above, the control information generator 32 may set a route that avoids bright areas or noisy areas. A route that avoids bright areas may be, for example, a route that avoids areas with many stores when the vehicle 1 travels at night, but is not limited to this. A route that avoids noisy areas may be, for example, a route that avoids areas near construction sites or areas with many people or cars, but is not limited to this.

[0072] Furthermore, the control information generator 32 may suppress external stimuli to the user by controlling devices within the vehicle 1 that affect the environment around the user. Fig. 9 is a diagram showing an example of the configuration of a vehicle 1c according to this embodiment, which controls devices within the vehicle 1c that affect the environment around the user. The vehicle 1c shown in Fig. 9 has a speaker 11, a sound reduction device 12, and a dimmer 13 added to the vehicle 1 of the first embodiment.

[0073] The speaker 11 outputs audio data output from an audio data output device (not shown) as sound. The audio data output device may be, for example, a device that plays music or the like within the vehicle 1c or a device that acquires broadcast or distributed audio data. The sound reduction device 12 is a device that reduces noise within the vehicle 1c, and may be a combination of a sound-absorbing sheet and a device that controls the insertion and removal of the sound-absorbing sheet, or a noise canceling device that detects noise within the vehicle 1c and generates sound in an opposite phase to the detected noise to cancel the noise through electrical processing, or other devices. The light dimmer 13 is a device that reduces light within the vehicle 1c, and may be, for example, a light control film that can adjust light transmittance and is installed on a rear window of the vehicle 1c, or a curtain that blocks light while the vehicle 1c is parked, or other devices.

[0074] In the example shown in FIG. 9 , in step S24 shown in FIG. 8 , the control information generator 32 further generates control information for operating the speaker 11, the sound reduction device 12, and the dimmer 13, and outputs the generated control information to the speaker 11, the sound reduction device 12, and the dimmer 13, thereby suppressing sound and light inside the vehicle 1. For example, the control information generator 32 generates control information for lowering the volume of the speaker 11 or turning the speaker 11 off, generates control information for causing the sound reduction device 12 to suppress sound inside the vehicle 1, and generates control information for causing the dimmer 13 to suppress light inside the vehicle 1. This makes it possible to suppress not only stimuli caused by driving but also stimuli caused by the surrounding environment other than driving, for the user. Note that in the example shown in FIG. 9 , an example is shown in which the vehicle 1c is equipped with the speaker 11, the sound reduction device 12, and the dimmer 13, but this is not limiting, and the vehicle 1c may be equipped with one or two of the speaker 11, the sound reduction device 12, and the dimmer 13.

[0075] Note that a speaker 11, a sound reduction device 12, and a dimmer 13 may be added to the vehicle 1a shown in Fig. 2 of the first embodiment, and the information processing device 3a may generate control information for controlling these devices and transmit it to the vehicle 1a via the communication unit 35. In this case, the vehicle 1a may also be equipped with one or two of the speaker 11, the sound reduction device 12, and the dimmer 13.

[0076] In addition, the information processing device 3 may perform both the operation of embodiment 2 and the operation described in this embodiment, and the vehicle 1a and information processing device 3a shown in Figure 2 may perform both the operation of embodiment 2 and the operation described in this embodiment.

[0077] Furthermore, although an example has been described here in which both stimuli caused by driving and stimuli caused by the user's surrounding environment are suppressed, vehicle 1c may also be configured to suppress only stimuli caused by the user's surrounding environment when the user is in a light sleep state.

[0078] Embodiment 4. In Embodiment 4, as in Embodiments 2 and 3, an example of driving control of a vehicle 1 by the information processing device 3 described in Embodiment 1 will be described. The configuration of the vehicle 1 in this embodiment is the same as that in Embodiment 1. Note that, although the description here takes as an example a configuration in which the information processing device 3 is provided inside the vehicle 1 as shown in FIG. 1 of Embodiment 1, the operation of this embodiment can also be similarly applied to the configuration example shown in FIG. 2 of Embodiment 1. Components having the same functions as those in Embodiment 1 are assigned the same reference numerals as those in Embodiment 1, and redundant description will be omitted.

[0079] FIG. 10 is a flowchart showing an example of a processing procedure in the information processing device 3 of this embodiment. Steps S1 and S2 are the same as steps S1 and S2 of embodiment 2. After step S2, the information processing device 3 predicts the wake-up time (step S31). Specifically, the estimation unit 31 predicts the user's wake-up time using the estimated sleep depth. For example, the estimation unit 31 predicts the wake-up time using the sleep depth estimation result and the standard human sleep pattern described in embodiment 1, and outputs the prediction result to the control information generation unit 32. Specifically, for example, the estimation unit 31 estimates the time of sleep onset and the sleep cycle using the sleep depth estimation result, calculates a time period in which REM sleep occurs within a certain period including the estimated time of arrival at the destination, and sets a time within the calculated time period as the predicted wake-up time. For example, the control information generation unit 32 may set the center time of the time period in which REM sleep occurs as the predicted wake-up time.

[0080] Next, the information processing device 3 sets a route for arriving at the destination within a certain time from the predicted wake-up time (step S32). Specifically, the control information generator 32 determines a route for the vehicle 1 to the destination so that the vehicle 1 arrives at the destination within a certain time from the wake-up time predicted in step S31, generates route information as control information indicating the determined route, and stores the route information in the information storage unit 34. Alternatively, the control information generator 32 may output the control information indicating the determined route to the driving control unit 33. By driving based on this route information, the vehicle 1 can arrive at the destination within a certain time from the predicted wake-up time. This increases the likelihood that the user will wake up naturally near the destination. In this way, even in this embodiment, driving control can be performed according to the user's sleep state.

[0081] In the above example, the wake-up time is predicted using the sleep depth, but this is not limited to this. The wake-up time may be set by the user, or may be estimated based on sleep information indicating at least one of the user's sleep history, the user's wake-up time history, and the user's planned wake-up time, as described below.

[0082] FIG. 11 is a diagram showing an example of the configuration of a vehicle according to this embodiment that estimates a wake-up time using sleep information. As shown in FIG. 11 , a vehicle 1d includes an information processing device 3d instead of the information processing device 3. The information processing device 3d includes a sleep information acquisition unit 38 in addition to the information processing device 3. The sleep information acquisition unit 38 acquires the sleep information by receiving it from an information providing device 6 that stores the user's sleep information, and outputs the acquired sleep information to the estimation unit 31. Note that FIG. 11 shows an example in which the information providing device 6 is a mobile terminal such as a smartphone, tablet, or personal computer carried by the user. The information providing device 6 is located inside the vehicle 1d, but the information providing device 6 may also be located outside the vehicle 1d. Furthermore, the information providing device 6 is not limited to a mobile terminal carried by the user, and may be a server device that manages the sleep information of multiple users.

[0083] The sleep information is, for example, information managed by a sleep app (application software) that acquires and manages information related to the user's sleep, but is not limited to this. For example, if the information providing device 6 has an alarm function, the wake-up time set in the alarm function may be used as the sleep information. Also, if the information providing device 6 has a schedule management function and the user registers a wake-up time in their schedule, the registered wake-up time may be used as the sleep information.

[0084] The sleep information managed by the sleep app includes, for example, the user's fall asleep time, wake-up time, and sleep cycle. When predicting the wake-up time using the sleep information managed by the sleep app, the estimation unit 31 may, for example, calculate an average wake-up time using the user's wake-up time history and use the calculated average as the predicted wake-up time. Alternatively, the estimation unit 31 may calculate an average sleep duration using the sleep onset time and wake-up time history, estimate the user's fall asleep using sensor information, and use the time that is the calculated average value after the estimated fall asleep time as the predicted wake-up time.

[0085] Furthermore, when the sleep information is a wake-up time set in the alarm function, the estimation unit 31 may use the wake-up time indicated by the sleep information as a predicted value for the wake-up time. Furthermore, when the sleep information is a wake-up time registered in a schedule, the estimation unit 31 may use the wake-up time indicated by the sleep information as a predicted value for the wake-up time. Note that when a schedule for a target day for wake-up time prediction is registered, the wake-up time in the schedule for the target day for wake-up time prediction is used as a predicted value for the wake-up time. When a schedule for a target day for wake-up time prediction is not registered, the estimation unit 31 may use a registered wake-up time for another day of the same week as a predicted value for the wake-up time, or the most common wake-up time among the registered wake-up times as a predicted value for the wake-up time.

[0086] The information processing device 3d shown in Fig. 11 is realized by, for example, the processing circuit exemplified in Fig. 3. The sleep information acquisition unit 38 is realized by the communication unit 103.

[0087] FIG. 12 is a flowchart showing an example of a processing procedure in the information processing device 3d of this embodiment that predicts a wake-up time using sleep information. Step S1 is the same as the example shown in FIG. 10. If the answer to step S1 is Yes, the information processing device 3d predicts a wake-up time using the sleep information (step S41). In detail, the estimation unit 31 predicts the user's wake-up time using the sleep information received from the sleep information acquisition unit 38 and outputs the prediction result to the control information generation unit 32. Step S32 is the same as the example shown in FIG. 10.

[0088] Note that a sleep information acquisition unit 38 may be added to the vehicle 1a or the information processing device 3a shown in FIG. 2 of the first embodiment, and the information processing device 3a may predict the wake-up time of the user based on the sleep information in the same manner as in the present embodiment, and set a route that will allow the user to arrive at the destination within a certain time from the predicted wake-up time.

[0089] In addition, the information processing device 3d may perform at least one of the operations of embodiment 2 and embodiment 3 and the operations described in this embodiment, and the vehicle 1a and information processing device 3a shown in Figure 2 may perform at least one of the operations of embodiment 2 and embodiment 3.

[0090] As described above, in this embodiment, the wake-up time is predicted and a route is set so that the user arrives at the destination within a certain time from the predicted wake-up time, which increases the likelihood that the user will wake up naturally near the destination and improves user comfort.

[0091] Fifth Embodiment Fig. 13 is a diagram showing an example of the configuration of a vehicle 1e according to a fifth embodiment. The vehicle 1e of this embodiment is similar to the vehicle 1 of the first embodiment except that it includes an information processing device 3e instead of the information processing device 3. Components having the same functions as those of the first embodiment are assigned the same reference numerals as those of the first embodiment, and redundant explanations will be omitted.

[0092] 13, the information processing device 3e is obtained by adding an identification unit 39 and a characteristic information storage unit 40 to the information processing device 3 of the first embodiment. Note that, although the following description will be given taking as an example a configuration in which the information processing device 3e is provided inside the vehicle 1e as shown in FIG. 1 of the first embodiment, the identification unit 39 and the characteristic information storage unit 40 may be added to the information processing device 3a in the configuration example shown in FIG. 2 of the first embodiment, and the operation of this embodiment may be performed.

[0093] In this embodiment, the sensor 2 includes, for example, a camera, and the sensor information includes image data acquired by capturing an image of a user with the camera. The sensor information acquisition unit 30 outputs the image data to the identification unit 39. The identification unit 39 identifies the user using the image data and outputs the identification result (user identification result) to the estimation unit 31. The user identification may be the identification of the individual user or the identification of the user's attributes. The attributes include, for example, at least one of gender and age, but are not limited to these.

[0094] The user is identified using, for example, facial recognition technology. When identifying an individual user, image data representing an image of the user's face is registered in advance, and the identification unit 39 identifies the individual user by comparing the registered image data with the photographed data. The age does not have to be the age itself, but may be indicated by a category such as infant, child, or adult, or by a generation category such as under 10 years old, teens, or twenties. The age category is not limited to these examples. In the example shown in FIG. 13 , the identification unit 39 identifies the user based on the photographed data, but the user may also be identified using vital data acquired as sensor information, or the user may be identified by combining the photographed data and vital data.

[0095] The characteristic information storage unit 40 stores characteristic information indicating sleep characteristics for each user or for each attribute. The characteristic information includes at least one of information indicating, for example, the user's sleep cycle, sleep duration, wake-up time, vital data characteristics for each sleep depth, characteristics of the user's body movements for each sleep depth, vital data characteristics of body movements during sleep and wakefulness, and characteristics of body movements during sleep and wakefulness, and is used for estimation by the estimation unit 31. The characteristic information may be information in a table format in which data for each item is stored for each user, or may be a trained model trained by machine learning as described below. When the characteristic information is in a table format, the characteristic information is calculated, for example, by statistically processing data accumulated in advance for each user or for each attribute. Furthermore, the characteristic information may be calculated based on sensor information acquired by the sensor 2 when the user previously rode in the vehicle 1e.

[0096] The estimation unit 31 estimates a sleep-related index using the identification result received from the identification unit 39 and the characteristic information stored in the characteristic information storage unit 40. The sleep-related index may be a sleep depth, a wake-up time, or a classification of whether or not the user is asleep, or may be two or more of these. That is, in the determination of step S1 shown in FIGS. 4 , 7 , 8 , 10 , and 12 , the estimation unit 31 may estimate whether or not the user is asleep using the identification result received from the identification unit 39 and the characteristic information stored in the characteristic information storage unit 40. In estimating whether or not the user is asleep, at least one of information indicating, for example, characteristics of vital data of body movements during sleep and wakefulness, characteristics of body movements during sleep and wakefulness, etc. is used as the characteristic information.

[0097] 4 , 7 , 8 , and 10 , the estimation unit 31 may estimate the sleep depth using the identification result received from the identification unit 39 and the characteristic information stored in the characteristic information storage unit 40. When estimating the user's sleep depth, the characteristic information used may include, for example, at least one of information indicating the user's sleep cycle, vital data characteristics for each sleep depth, and body movement characteristics for each sleep depth. When estimating the wake-up time described in the fourth embodiment, the estimation unit 31 may estimate the sleep depth using the identification result received from the identification unit 39 and the characteristic information stored in the characteristic information storage unit 40. When estimating the user's wake-up time, the characteristic information used may include, for example, at least one of information indicating the sleep duration, the wake-up time, and the like. In this case, the characteristic information may be an actual value of at least one of the user's wake-up time and the sleep duration.

[0098] As described above, the characteristic information may be a trained model. FIG. 14 is a diagram showing an example configuration of the estimation unit 31 according to the present embodiment when estimation is performed using machine learning. In the example shown in FIG. 14 , the estimation unit 31 includes a learning unit 311 and an inference unit 312. The learning unit 311 generates a trained model through supervised learning using, for example, multiple data sets each including input data as feature amounts and corresponding indexes related to sleep as ground truth data, and stores the generated trained model as characteristic information in the characteristic information storage unit 40. For example, the feature amounts include information calculated from sensor information.

[0099] When the trained model is a model for inferring whether or not a user is asleep, the input data includes, for example, at least one of information indicating characteristics of vital data of body movements during sleep and wakefulness, characteristics of body movements during sleep and wakefulness, etc. When the trained model is a model for inferring sleep depth, the input data includes, for example, at least one of information indicating the user's sleep cycle, characteristics of vital data for each sleep depth, and characteristics of the user's body movements for each sleep depth, etc. The input data may be calculated from data previously acquired by the sensor 2, may be calculated from data acquired by a device other than the sensor 2, or may be a mixture of these. The input data may also be acquired by a sleep app or the like. Note that the correct answer data may be determined, for example, using electroencephalograms acquired separately, or may be determined based on the results of providing a stimulus to the user, or may be determined by other methods.

[0100] Any supervised learning algorithm may be used to generate the trained model in the learning unit 311, but for example, a neural network model may be used. The neural network is composed of an input layer consisting of multiple neurons, an intermediate layer (hidden layer) consisting of multiple neurons, and an output layer consisting of multiple neurons. The intermediate layer may be one layer, or two or more layers.

[0101] Figure 15 is a schematic diagram showing an example of a neural network. For example, in a three-layer neural network as shown in Figure 15, when multiple inputs are input to the input layer (X1-X3), the values ​​are multiplied by weight W1 (w11-w16) and input to the intermediate layer (Y1-Y2), and the result is further multiplied by weight W2 (w21-w26) and output from the output layer (Z1-Z3). This output result varies depending on the values ​​of weight W1 and weight W2.

[0102] In this embodiment, the weights W1 and W2 are adjusted so that the output from the output layer when a feature is input approaches the sleep-related index, which is the correct data, to learn the relationship between the feature and the correct data. Note that the machine learning algorithm is not limited to a neural network and may be other algorithms such as a support vector machine. Furthermore, the machine learning used to generate the trained model in the learning unit 311 is not limited to supervised learning and may be reinforcement learning or the like.

[0103] The inference unit 312 calculates feature amounts using the sensor information received from the sensor information acquisition unit 30, and infers sleep-related indices by inputting the feature amounts into the trained model stored in the characteristic information storage unit 40. In this way, sleep-related indices are estimated.

[0104] The trained model may be generated for each individual user or for each attribute. For example, a trained model may be generated for each age group. When the identification result is an identification result for an individual user, the estimation unit 31 infers a sleep-related index using a trained model corresponding to the individual user. Furthermore, when the identification result is an identification result for a user attribute, the estimation unit 31 infers a sleep-related index using a trained model corresponding to the attribute. Alternatively, instead of generating a trained model for each individual user or for each attribute, the individual user's identification information or attributes may also be input as features to generate a trained model, and the individual user's identification information or attributes may also be input as features to the trained model during inference.

[0105] In addition, Figure 14 shows an example in which the estimation unit 31 is equipped with a learning unit 311, but this is not limited to this. The vehicle 1e may be equipped with a learning device that generates a learned model separately from the vehicle 1e, the learning device may generate the learned model, and the learned model generated by the learning device may be stored in the characteristic information storage unit 40.

[0106] The information processing device 3e shown in Fig. 13 is realized by, for example, the processing circuit exemplified in Fig. 3. The identification unit 39 shown in Fig. 13 is realized by the processor 101 shown in Fig. 3 executing a program stored in the memory 102 shown in Fig. 3. The memory 102 shown in Fig. 3 is also used to realize the identification unit 39 shown in Fig. 13. The characteristic information storage unit 40 shown in Fig. 13 is part of the memory 102 shown in Fig. 3.

[0107] The operation of this embodiment is applicable when estimating a sleep-related index in any one of the first to fourth embodiments or in a combination of two or more of the first to fourth embodiments.

[0108] As described above, in this embodiment, an individual user or a user's attributes are identified, and based on the identification results, sleep-related indices such as whether the user is asleep, sleep depth, and wake-up time are estimated. This provides the same effects as in the first embodiment, and also improves the accuracy of estimating sleep-related indices.

[0109] The configurations shown in the above embodiments are merely examples, and may be combined with other known technologies, or different embodiments may be combined with each other. It is also possible to omit or modify parts of the configurations as long as they do not deviate from the gist of the invention.

[0110] 1, 1a, 1b, 1c, 1d, 1e Vehicle, 2 Sensor, 3, 3a, 3b, 3d, 3e Information processing device, 4 Traveling mechanism, 5 Awakening promotion device, 6 Information providing device, 11 Speaker, 12 Sound reduction device, 13 Light reduction device, 30 Sensor information acquisition unit, 31 Estimation unit, 32 Control information generation unit, 33 Traveling control unit, 34 Information storage unit, 35, 36 Communication unit, 37 Input acceptance unit, 38 Sleep information acquisition unit, 39 Identification unit, 40 Characteristic information storage unit, 100 Vehicle system, 311 Learning unit, 312 Inference unit.

Claims

1. An information processing device comprising: an estimation unit that estimates a sleep depth of a user using sensor information acquired by a sensor that detects at least one of the state of the user riding in a vehicle and the environment surrounding the user; and a control information generation unit that generates control information used to control the autonomous driving of the vehicle using the sleep depth estimated by the estimation unit.

2. The information processing device described in claim 1, characterized in that the estimation unit estimates the sleep depth for a certain time ahead, and the control information generation unit determines a period during which the user is predicted to be in light sleep using the sleep depth for the certain time ahead estimated by the estimation unit, determines a route for the vehicle so as to arrive at a parking location within the determined period, and generates the control information indicating the determined route.

3. The information processing device according to claim 2, characterized in that the control information generation unit generates the control information for parking the vehicle in the parking location, and, after the vehicle arrives at the parking location, if the estimation unit determines that the user is in a deep sleep using the estimation result of the sleep depth estimated based on the sensor information, generates the control information for causing the vehicle to depart from the parking location and resume driving.

4. The information processing device according to claim 2 or 3, characterized in that the parking location includes at least one of a service area and a parking area.

5. An information processing device as described in any one of claims 1 to 4, characterized in that when the control information generation unit determines that the user is in a light sleep using the sleep depth estimated by the estimation unit, it generates the control information to make the vehicle's driving gentler compared to normal driving without restrictions.

6. The information processing device according to claim 5, wherein the gentle driving includes driving on a route with few bumps and irregularities.

7. The information processing device according to claim 5 or 6, characterized in that the gentle driving includes driving on a route with few sharp curves.

8. The information processing device according to any one of claims 5 to 7, characterized in that the gentle driving includes driving in which at least one of the acceleration and the speed of the vehicle is equal to or lower than a threshold value.

9. An information processing device as described in any one of claims 1 to 8, characterized in that when the control information generation unit determines that the user is in a light sleep using the sleep depth estimated by the estimation unit, it operates a sound reduction device to reduce noise inside the vehicle.

10. An information processing device as described in any one of claims 1 to 9, characterized in that when the control information generation unit determines that the user is in light sleep using the sleep depth estimated by the estimation unit, it operates a dimming device to reduce light inside the vehicle.

11. An information processing device as described in any one of claims 1 to 10, characterized in that the estimation unit predicts the wake-up time of the user using the estimated sleep depth, and the control information generation unit determines a route for the vehicle so as to arrive at the destination within a certain time from the wake-up time predicted by the estimation unit, and generates the control information indicating the determined route.

12. An information processing device as described in claim 11, comprising: an identification unit that identifies the user; and a characteristic information acquisition unit that acquires characteristic information indicating sleep-related characteristics for each user, wherein the estimation unit predicts the wake-up time using the identification result by the identification unit and the characteristic information.

13. The information processing device according to claim 12, wherein the characteristic information includes a performance value of at least one of the wake-up time and the sleep duration of the user.

14. An information processing device as described in any one of claims 1 to 13, comprising: an identification unit that identifies the user; and a characteristic information acquisition unit that acquires characteristic information indicating sleep-related characteristics for each user, wherein the estimation unit estimates the sleep depth using the identification result by the identification unit and the characteristic information.

15. An information processing device as described in any one of claims 1 to 14, characterized in that it comprises an identification unit that identifies attributes of the user, and a characteristic information acquisition unit that acquires characteristic information indicating sleep-related characteristics for each of the attributes, wherein the estimation unit estimates the sleep depth using the attributes and characteristic information identified by the identification unit.

16. An information processing device as described in claim 14 or 15, characterized in that the characteristic information is a trained model generated by supervised learning using multiple data sets consisting of features calculated using the sensor information and corresponding correct data of sleep depth.

17. A vehicle capable of autonomous driving, comprising: an estimation unit that estimates a sleep depth of a user using sensor information acquired by a sensor that detects at least one of the state of a user riding in the vehicle and the environment surrounding the user; a control information generation unit that generates control information used to control the driving of the vehicle using the sleep depth estimated by the estimation unit; and a driving control unit that controls the autonomous driving of the vehicle using the control information.

18. A vehicle control method in an information processing device that controls a vehicle capable of autonomous driving, comprising: a step of estimating a sleep depth of a user using sensor information acquired by a sensor that detects at least one of the state of a user riding in the vehicle and the environment surrounding the user; and a step of generating control information to be used for controlling the autonomous driving of the vehicle using the estimated sleep depth.

19. A program that causes a computer system that controls a vehicle capable of autonomous driving to execute the following steps: estimating the depth of sleep of a user using sensor information acquired by a sensor that detects at least one of the state of a user riding in the vehicle and the environment surrounding the user; and generating control information used to control the autonomous driving of the vehicle using the estimated depth of sleep.

Images